Shedding Some Light

Sorry for the delay post, but I have been a bit busy here in Arizona at the Advisor Summit. I am here as a speaker for the first time.

To understand this conference completely, you might not know there are really six conference going on at the same time. VFP, MS Access, MS SharePoint, SQL Server/.NET, Lotus, and Novell GroupWise.

VFP, Access, SharePoint, GroupWise each have two tracks, and SQL Server/.NET has one track. Lotus has three tracks.

The VFP Keynote was given by Alan Griver. Alan showed off some of the Sedna components released in the Community Technology Previews (CTP) as well as some of the reporting changes expected for the VFP 9 SP2, and the some of the community happenings like the VFPX project (referencing the GDI+ project headed by Bo Durban) and promotional videos generated by the community. He opened the session with a tribute to Drew and Brent Speedie. He showed a couple of Kevin Ragsdale’s promotional videos and some cool GDI+ examples developed by Cesar Chalom. Overall it was is good keynote. Not a lot of flash of past years, but I would say there was more concrete examples than PowerPoint fluff. Definitely more on direction than marketing junk we have seen at more recent conferences. Overall, I’d rank it a 4 out of 5 stars.

Based on the attendance at the Keynote, we are estimating there are between 100 and 120 VFP attendees at the conference. This is an estimate and nothing official.

Next I attended Craig Boyd’s How to Build .NET ActiveX Controls for Use in VFP session. Craig showed how you can simply build controls in VB.NET or C#.NET to add more modern UI controls to your application. Honestly, it looks pretty easy, but only because Craig did all the hard work for us by building a recipe for us to follow. I am looking forward to reading his whitepaper on this topic. Overall, I’d rank it 5 stars.

Next session was Jim Duffy’s SQL Server 2005 Express and VFP. I only wanted one thing out of this session which is the installation and initial configuration of SQL Server Express. Jim took a risk and scrapped his install and started from scratch in the presentation. His knowledge on the subject was good and he delivered on the things I wanted from the session. There was not much VFP in the session, but I think the overall value was extremely high for developers considering upgrading from MSDE. The best part of the session was the comparison between the different versions of Express (which I was not aware of), and I learned the SQL Agent is not included in Express as it was in MSDE. So I guess I gain Vista compatibility, but now I have to build code to schedule backups for my clients. Overall – 5 stars.

Lunch was good – Mexican.

I skipped the first session of the afternoon because I saw both Tamar Granor’s Customizing the VFP Property Sheet session and Cathy Pountney’s Advanced Reporting in VFP 9 session. Instead I rehearsed the material for my Use and Extend the Visual FoxPro Data Explorer session, which was next. According to the evals, it was well received. This session was the first time anyone outside of Microsoft has seen some of the potential changes coming in the Sedna version of the Data Explorer.

The last breakout session of the day was Craig Boyd’s Windows Vista and VFP. Craig spent a lot of time showing us Vista and the capabilities of Vista. Specifically he concentrated on feature with APIs VFP developers can hook into. I think the API to the desktop search will be something very cool and beneficial to business applications. Craig also showed us VFP 9 SP1 (not any previews of SP2 which is suppose to address Vista issues) and how well it is already working and some of the glitches he has experienced with the Report Preview window. Great session that got me thinking about some of the future possibilities. After all, the Vista compatibility is my number one feature of SP2 and Sedna. Five stars.

There was a good reception after the break out sessions. The food was excellent and the discussions fun. Ended up outside with Rainer Becker, Craig Boyd, and Kevin (sorry, I have forgotten his last name). We discussed VFP related stuff and hassled the waitress who delivered cold food the night before until 12:30 or so. Good times and a sore stomach muscles from laughing so much. A great day.

Congratulations goes out today to Dan Taylor of PowerChurch Software. Dan’s entry was randomly drawn from the folks who registered for Southwest Fox 2006 in the first 15 days of August. Dan is the winner of the $300 scholarship from White Light Computing. This is the first door prize we are giving for Southwest Fox 2006.

This is going to be Dan’s first trip to Southwest Fox and he is very excited about going.

Again, congratulations Dan. I look forward to meeting you in Arizona.

Thanks to all who have registered for the conference. Keep them coming. There is no better time to get your registration processed than right now. The conference is only 63 days away!

(Yikes, I better get my session materials finished up)

I found one more disaster recovery issue with the loss of my hard drive.

Today is administrative day here at White Light. Every Monday morning I cut expense checks, process invoices, do a little marketing, correspond with a couple of prospects, send monthly reports to the accountant, etc. It is a way to get all the stuff I least enjoy about my job done before I have fun for the next 80 hours or so.

I ran my bank reconciliation report after balancing my checkbook. All the reports I generate out of Quickbooks get printed to Acrobat 7.0 so I have a permanent electronic version (naturally backed up for offsite storage). The report displayed in the viewer like it always does, but today I was prompted by a “your system has changed, please activate or this product will no longer work” message. Fortunately I was connected to the Internet and all went well with the activation.

But what would have happened if I was not connected? What if I was testing some reports in my applications while I was sitting on the beach at my in-laws cottage? What if I was at a client’s office and running some reporting scenarios, or they asked me to print an invoice while I was visiting? Would I have been hosed? I think so. Sort of reminds me of the stories where someone is on a plan about to use one of the Microsoft Office apps and it prompts them with a surprise activation sequence. Don’t have your CD or DVD with you? Bummer.

So the question begs: what does Adobe know about me and my machine?

Looks like another case where the pirates have won. I understand the need for activation, but in the long term it really only hurts those of us who are honestly licensing our software. The pirates know all the hacks and where to go to get the ones they have not used.

This week I was able to test part of the White Light Computing disaster recovery plan for my primary development laptop. I experienced the siuation I believe every computer user fears most with their machines: hard drive failure.

Wednesday evening I returned home from one of my clients. I pulled my laptop out of my backpack and put it on my desk. It started whining and screeeeeching, and making the horrific clicking sound you hear when the drive is beginning to fail. I was very fortunate in the fact that the drive did not immediately die, and I was able to get it to stop making the bad noices by placing it absolutely level on the desk (in case you are wondering, I normally place it in a port replicator on a stand on the desk which is not level). If I picked it up it would click and screech.

I immediately connected it to my backup drive. I am using two Western Digital 250MB USB 2.0 drives to back up each of the data partitions using Powerquest’s DriveImage (now owned by Symantec and the technology rolled into Ghost 10). I ran the back up jobs normally scheduled each night. I just backed up the C: partition on Saturday and have not loaded any new programs so I figured I was okay on this.

Thursday morning I called HP support and reported the problem. The young lady was nice enough to listen to the clicking and screeching, but had to follow procedure to get my drive replaced. I told her I was willing to do so as long as none of the steps was FORMAT C-colon. The BIOS drive diagnostics showed nothing. I wasted more than an hour running these which is time lost forever. I told her I was going to buy a replacement drive anyway and needed the drive specs. She offered to sell me a 80GB drive for US$628. I asked her if she was offering me a new laptop for that price.

I picked up a new 120GB drive for US$200 at the local CompUSA. It took me 15 minutes to install it. I had some work to do at one of my clients (who is kind enough to provide me a boat anchor to work on when I am in their office). I was hoping to restore the drive while working on their conversion code. For some reason I could not figure out how restore the images from the DVDs I burned and I did not think to bring my USB drive.

I was able to handle all my email via the Web interfaces. I realized at the time I did not have access to any of my email templates I use regularly. I have five email templates used to correspond with developers who buy my developer tools, and a couple more for reporting hours to clients. I also have several templates to answer common questions on the various FoxPro forums. This was failure point number one. To fix this I will be putting the templates on a thumb drive.

Thursday night was our monthly DAFUG meeting. Cathy Pountney visited and rehearsed her testing session. It was a great meeting, but it delayed the reload. I got home after 10:00 and spent some time with the family. They were watching Harry Potter 2 and I got sucked in.

The reload started at 11:30. I was anticipating each partition restore to take a little less than it takes to back the partition up because I was verifying the image before loading. The 25GB C: partition took less than an hour (it takes close to 2 hours to back up). I did not watch it and was planning on it running overnight. I could not sleep so I checked it and it was already done. I kicked off the 15GB D: partition that took fifteen minutes, and the 30GB J: partition took a little more than a half hour.

Rebooted and the machine started just like it did before the drive crash. SWEET!

I had to change the drive letter for the J: partition as it was assigned to F:. I had to reactivate the IntelliSync software (used to sync my Treo phone). I was half expecting to have to reactivate Windows XP because the drive size changed, but it did not ask me to do so.

The one thing I was surprised about is Zone Alarm lost its marbles. I had to go through the cycle of approving Internet access for several applications. This is not a big deal, but aggravating. So action item number two is to export out my Zone Alarm settings periodically.

I had the machine in working order by 3:00am including downloading emails, checking a couple of newsgroups, and letting FeedDemon refresh the RSS feeds.

The only other problem I have experienced is files saved to the desktop. Under my partition backup scheme, these files are only backed up once a month when I image the C: partition. I happened to create a spreadsheet Monday morning and saved it to the desktop. This is really rare for me, but it pointed out a hole in the disaster recovery plan. It only took me a couple of minutes to recreate the spreadsheet and in this case I still have a working drive I could have recovered it from. If the drive was completely dead I could have lost nearly a month of work. Action item three is to ensure the Documents and Settings folder gets backed up more frequently and I only have shortcuts on the desktop.

I lucked out. The situation could have been catastrophic. It was not. I did lose much of a day, but was still able to deliver some work for one of my clients. I had to delay another client for something I was hoping to deliver Thursday, but he was very understanding.

I am going to buy a second 120GB drive later today and go through this process on a regular basis. It was an excellent way for me to test the recover process. The thing that slowed me down was the time to go get the drive and the time to restore the images. If I had the drive I literally would have been up and running in a couple of hours.

I only tested out part of my disaster recovery. What if my machine died completely? I have a spare laptop (older but still working as a test machine) and the data images I can use to restore the latest files as apposed to restoring the partition. Now if a fire destroyed both laptops I would be in trouble, but this is a risk I am managing.

The questions of the day are:

1. Do you have a disaster recovery plan?
2. How good is your disaster recovery plan?
3. How long would it take you to get back in business?
4. Would you be able to recover all your data?

I am very happy with the way DriveImage saved my day. I have used it before to restore files, but never to restore entire partitions. Yes, I never really fully tested my disaster recovery! I fell into the 95 percentile who don’t. I am ashamed to admit this, but I can also say I am part of the luck few (maybe the 5 percentile) who survived despite not doing the appropriate testing.

So how well would you survive losing the drive on your primary development machine?

Yes, I just read a post on a Microsoft blog which states: “Yes it’s true: MSDE will not be supported on Windows Vista“

SET RANT ON && My apologies

Put this in the category of “What the heck are they thinking!”

I am not sure who the powers that be are with respect to the decision, but I think the developer community needs to speak up on this one. I for one will be emailing Milind Lele (VFP Project Manager) about this. For years Microsoft has pushed the VFP developer and even the .NET developer towards SQL Server and MSDE 2000. For them to pull the rug out on us like this is a load of baloney. I applauded Microsoft when they included Vista (at the time referenced as Longhorn) compatibility on the VFP Sedna Roadmap. At the time I noted this was my personal number one request for the next version of VFP. The August 2006 letter from Milind indicates the Fox Team is hard at work testing this compatibility. I am excited.

So now the SQL Server team decides it is in my clients’ best interest to use SQL Server Express? I might agree that this is a long term goal, but I have not certified my apps to do this, and in fact have struggled with SQL Server Express to be cooperative. I believe the “high point” reasons are not enough. Come on, Microsoft Update is a big reason? I take care of my clients: don’t be concerned with this Microsoft.

The support life cycle is a minor problem in my opinion, but significant enough to consider. But honestly, if the SQL Server team was forced to fix any Vista incompatibilities with a service pack then Microsoft would have to bump the life cycle expiration (if they follow with the same action they have done for other products). The customers (us and the clients we service) would be big winners.

I understand the benefits of moving on up (except for the seriously slow Management Studio Express applet), but there are cold realities in this world of business and it is not always the practical decision to “upgrade.”

I hope Microsoft understands why I will not be recommending Vista to my customers who need and demand MSDE. I guess I can continue to load it on a Windows XP Pro machine, or even the “clunky” Windows 2000 machine sitting in the closet for those clients who want to get Vista-ized on their new desktops. But the clients who use MSDE are usually in a small office and I will just have to convince them to keep an older OS around. Or maybe I will have to recommend another backend database (Whil and Ted and the rest of the MySQL proponents need not comment {g}).

This is just one bonehead decision, and one I hope we can rise up and influence to change. It has been done before so the trail has been blazed. Get those emails sent.

I will post my findings as I learn more.

*< RAS 9-Aug-2006, keep commented until this problem is debugged.
*< SET RANT OFF

Recently I blogged about data overflow not always happening with numeric data in Visual FoxPro. This was a frustrating problem to track down and yesterday I ran into another side effect of this problem when using TRANSFORM().

If you execute the following in the Command Window:

?TRANSFORM(6500.0000/0.00015)

You get 43333333.333333340 dumped on the VFP desktop.

If you try to store the result of 6500.0000/0.00015 into a N(12,5) column in a table you get 43333333.333 even though one could expect an overflow condition to occur. But as I relearned a couple of weeks ago, this is not the case.

I was debugging this same conversion I originally discovered this behavior with last time for a different overflow problem. In the text output file I was seeing the asterisks for the column storing the large number. In the intermediate and temporary DBF of the output I was seeing 43333333.333. This particular field is numeric in nature, but I pass it along to the receiving system as a character field per the spec. To convert the number I elected to use TRANSFORM() because they did not want any leading spaces or trailing zeros. The STR() function returns asterisks by default on the large number and requires me to specify the number of digits and the decimals to avoid the overflow representation. TRANSFORM() was not only easier and required less code, but also met the exact requirements for this situation.

But yesterday I was seeing stars. Stepping through the code I was observing the values without overflow until I hit the code to TRANSFORM() the column to a character representation. The result was the overflow asterisks. Argh.

The VFP Help file indicates their can be problems with large negative numbers not getting correctly transformed, but says nothing about large positive numbers. What is happening here is that VFP realizes the overflow condition in the table column and transforms it correctly to asterisk.

The problem was tricky to figure out and time consuming to track down, but thanks to the new CAST() function the solution was easy. I casted the field early in the conversion process to ensure the column is wide enough. The real kicker is the data was bad data (not that I am being judgemental {g}).

Yesterday was a pretty frustrating day all the way around, and this was just the frosting on the cake.

I just got off the phone with Bob Kocher who tirelessly works on the Southwest Fox Conference every year so we can all come to Arizona and have a ball, and learn about our favorite development tool – Visual FoxPro. Bob and Sharon put on a fantastic conference as many of you already know from reading my blog, other blogger’s posts, the UT Conference notes, and posts on the various Fox forums.

I am very passionate about going to conferences. I feel they are the single biggest way one can jump start a career advancement move. It is a way to rejuvenate your spirit and it brings fresh ideas instantly to your current projects. I feel everyone should attend at least one conference a year. I also feel Southwest Fox is a great conference!

I made Bob an offer this evening to back up my passion. This is the first time Therese and I have done this. I am proud to announce that White Light Computing is offering one scholarship to attend Southwest Fox 2006. This is a US$300.00 scholarship (half the current “tuition” to register for this year’s event in October).

So how do you qualify for this? Simple, register for Southwest Fox 2006 between now and August 15, 2006. White Light Computing and Southwest Fox will randomly draw one name from the list of attendees who register between August 1, 2006 (today) and August 15, 2006 after the registrations for those days are processed.

This means you have to pay for the conference now. White Light will send you a check in the mail (or to a PayPal account) for US$300.00 before the end of the month (definitely before your credit card bill is due).

Just a few rules and regulations and notes:

1. One name will be drawn.
2. The winner will be notified by a phone call from Rick Schummer.
3. The conference registration cannot be cancelled. You may send another developer/person to take your place, but the registration cannot be withdrawn.
4. You must be 18 years of age to be eligible.
5. Except where prohibited, participation in the scholarship constitutes winner’s consent to White Light Computing and Southwest Fox to use winner’s name, likeness, opinions, biographical information, hometown and state for promotional purposes in any media, worldwide, without further payment or consideration.
6. Registrations must be processed before midnight (Mountain Time) of August 15, 2006.
7. Each full conference registration is one entry in the drawing.
8. Odds of winning will depend on the number of registrations during the allowed period.
9. The judges rulings are final.
10. All tax considerations are the responsibility of the winner (consult your tax accountant for specific details).
11. No employees of White Light Computing, or any family members of the employees of White Light Computing are eligible.
12. Your mileage may vary, some settlement may occur during shipping.

Your boss is giving you a hard time? Let me know. I am willing to talk or email each and every one of them to help you get the okay to head to any FoxPro conference. I am totally serious. I have fought numerous times with my old bosses to get myself to a conference as well as teammates. I have a way of getting them to see the light!

White Light Computing is having a fantastic 2006! This is just a little way Therese and I are going to pay it forward!

So what are you waiting for, get moving, tell your Fox friends, get registered for the best conference of the year!

Good Luck!

Anyone who has worked with VFP data will tell you the DBF is a fixed record length and structures do not have variable length columns, even with the introduction of data types like VarChar. A VarChar is still a fixed length field in the table structure, but internally VFP treats it differently when you work with the data. If you design a numeric field 5,2 you have two significant digits, a decimal place, and two decimal places. Right??

Yes, but only most of the time. {g}

Today I ran into a situation that probably has happened to me only once before in more than 17 years of working with DBF files. I am working on a death march conversion. Part of the conversion does some mathematical calculations with cost. The columns we are fitting data into are N(5,2) so any information 99.99 or less should fit just fine. We ran the conversion and the client was complaining about data being off by 2 cents. We figured there was some rounding problem with the data. My co-worker on the project watched the process in the debugger. The data in the SCATTER NAME object had 78 cents (.78). The next line executed is an INSERT INTO … NAME and in the column was 80 cents (.80).

I was asked by my co-worker to confirm he was seeing what he thought crazy behavior in VFP. I asked him to BROWSE the table. Sure enough it was true. This is sort of what we saw in a column with two decimal places of precision.

89.54
98.29
05.78
147.8

If it was a rounding problem we would have seen several of the values round to the tenth. What caught my eye was the missing hundredths value in the last and current record. At worse I expected .80, but in this case we saw a three whole digits and one decimal place stuffed into a N(5,2) column.

Argh!

Visual FoxPro is dynamically interpreting the structure a different way. I think it should creat an overflow condition, which immediately sticks out like a sore thumb. Instead, we lost a couple of hours of time tracking this down.

It is not until I enter in a numeric field with more than 5 significant digits that I get “Numeric overflow. Data was lost” (Error 39). Interesting behavior for sure. I will not be reporting this as a bug because it is probably the way DBFs have worked since the beginning of time, but it sure can lead to some expensive debugging sessions if you are not aware of this behavior.

I wrote a short test program to see if this is an issue with a SCATTER NAME object vs. doing a standard APPEND BLANK and a REPLACE, but the behavior is the same.

Hopefully this blog entry will save you some time with this problem down the road.