Shedding Some Light

I found one more disaster recovery issue with the loss of my hard drive.

Today is administrative day here at White Light. Every Monday morning I cut expense checks, process invoices, do a little marketing, correspond with a couple of prospects, send monthly reports to the accountant, etc. It is a way to get all the stuff I least enjoy about my job done before I have fun for the next 80 hours or so.

I ran my bank reconciliation report after balancing my checkbook. All the reports I generate out of Quickbooks get printed to Acrobat 7.0 so I have a permanent electronic version (naturally backed up for offsite storage). The report displayed in the viewer like it always does, but today I was prompted by a “your system has changed, please activate or this product will no longer work” message. Fortunately I was connected to the Internet and all went well with the activation.

But what would have happened if I was not connected? What if I was testing some reports in my applications while I was sitting on the beach at my in-laws cottage? What if I was at a client’s office and running some reporting scenarios, or they asked me to print an invoice while I was visiting? Would I have been hosed? I think so. Sort of reminds me of the stories where someone is on a plan about to use one of the Microsoft Office apps and it prompts them with a surprise activation sequence. Don’t have your CD or DVD with you? Bummer.

So the question begs: what does Adobe know about me and my machine?

Looks like another case where the pirates have won. I understand the need for activation, but in the long term it really only hurts those of us who are honestly licensing our software. The pirates know all the hacks and where to go to get the ones they have not used.

This week I was able to test part of the White Light Computing disaster recovery plan for my primary development laptop. I experienced the siuation I believe every computer user fears most with their machines: hard drive failure.

Wednesday evening I returned home from one of my clients. I pulled my laptop out of my backpack and put it on my desk. It started whining and screeeeeching, and making the horrific clicking sound you hear when the drive is beginning to fail. I was very fortunate in the fact that the drive did not immediately die, and I was able to get it to stop making the bad noices by placing it absolutely level on the desk (in case you are wondering, I normally place it in a port replicator on a stand on the desk which is not level). If I picked it up it would click and screech.

I immediately connected it to my backup drive. I am using two Western Digital 250MB USB 2.0 drives to back up each of the data partitions using Powerquest’s DriveImage (now owned by Symantec and the technology rolled into Ghost 10). I ran the back up jobs normally scheduled each night. I just backed up the C: partition on Saturday and have not loaded any new programs so I figured I was okay on this.

Thursday morning I called HP support and reported the problem. The young lady was nice enough to listen to the clicking and screeching, but had to follow procedure to get my drive replaced. I told her I was willing to do so as long as none of the steps was FORMAT C-colon. The BIOS drive diagnostics showed nothing. I wasted more than an hour running these which is time lost forever. I told her I was going to buy a replacement drive anyway and needed the drive specs. She offered to sell me a 80GB drive for US$628. I asked her if she was offering me a new laptop for that price.

I picked up a new 120GB drive for US$200 at the local CompUSA. It took me 15 minutes to install it. I had some work to do at one of my clients (who is kind enough to provide me a boat anchor to work on when I am in their office). I was hoping to restore the drive while working on their conversion code. For some reason I could not figure out how restore the images from the DVDs I burned and I did not think to bring my USB drive.

I was able to handle all my email via the Web interfaces. I realized at the time I did not have access to any of my email templates I use regularly. I have five email templates used to correspond with developers who buy my developer tools, and a couple more for reporting hours to clients. I also have several templates to answer common questions on the various FoxPro forums. This was failure point number one. To fix this I will be putting the templates on a thumb drive.

Thursday night was our monthly DAFUG meeting. Cathy Pountney visited and rehearsed her testing session. It was a great meeting, but it delayed the reload. I got home after 10:00 and spent some time with the family. They were watching Harry Potter 2 and I got sucked in.

The reload started at 11:30. I was anticipating each partition restore to take a little less than it takes to back the partition up because I was verifying the image before loading. The 25GB C: partition took less than an hour (it takes close to 2 hours to back up). I did not watch it and was planning on it running overnight. I could not sleep so I checked it and it was already done. I kicked off the 15GB D: partition that took fifteen minutes, and the 30GB J: partition took a little more than a half hour.

Rebooted and the machine started just like it did before the drive crash. SWEET!

I had to change the drive letter for the J: partition as it was assigned to F:. I had to reactivate the IntelliSync software (used to sync my Treo phone). I was half expecting to have to reactivate Windows XP because the drive size changed, but it did not ask me to do so.

The one thing I was surprised about is Zone Alarm lost its marbles. I had to go through the cycle of approving Internet access for several applications. This is not a big deal, but aggravating. So action item number two is to export out my Zone Alarm settings periodically.

I had the machine in working order by 3:00am including downloading emails, checking a couple of newsgroups, and letting FeedDemon refresh the RSS feeds.

The only other problem I have experienced is files saved to the desktop. Under my partition backup scheme, these files are only backed up once a month when I image the C: partition. I happened to create a spreadsheet Monday morning and saved it to the desktop. This is really rare for me, but it pointed out a hole in the disaster recovery plan. It only took me a couple of minutes to recreate the spreadsheet and in this case I still have a working drive I could have recovered it from. If the drive was completely dead I could have lost nearly a month of work. Action item three is to ensure the Documents and Settings folder gets backed up more frequently and I only have shortcuts on the desktop.

I lucked out. The situation could have been catastrophic. It was not. I did lose much of a day, but was still able to deliver some work for one of my clients. I had to delay another client for something I was hoping to deliver Thursday, but he was very understanding.

I am going to buy a second 120GB drive later today and go through this process on a regular basis. It was an excellent way for me to test the recover process. The thing that slowed me down was the time to go get the drive and the time to restore the images. If I had the drive I literally would have been up and running in a couple of hours.

I only tested out part of my disaster recovery. What if my machine died completely? I have a spare laptop (older but still working as a test machine) and the data images I can use to restore the latest files as apposed to restoring the partition. Now if a fire destroyed both laptops I would be in trouble, but this is a risk I am managing.

The questions of the day are:

1. Do you have a disaster recovery plan?
2. How good is your disaster recovery plan?
3. How long would it take you to get back in business?
4. Would you be able to recover all your data?

I am very happy with the way DriveImage saved my day. I have used it before to restore files, but never to restore entire partitions. Yes, I never really fully tested my disaster recovery! I fell into the 95 percentile who don’t. I am ashamed to admit this, but I can also say I am part of the luck few (maybe the 5 percentile) who survived despite not doing the appropriate testing.

So how well would you survive losing the drive on your primary development machine?

Yes, I just read a post on a Microsoft blog which states: “Yes it’s true: MSDE will not be supported on Windows Vista“

SET RANT ON && My apologies

Put this in the category of “What the heck are they thinking!”

I am not sure who the powers that be are with respect to the decision, but I think the developer community needs to speak up on this one. I for one will be emailing Milind Lele (VFP Project Manager) about this. For years Microsoft has pushed the VFP developer and even the .NET developer towards SQL Server and MSDE 2000. For them to pull the rug out on us like this is a load of baloney. I applauded Microsoft when they included Vista (at the time referenced as Longhorn) compatibility on the VFP Sedna Roadmap. At the time I noted this was my personal number one request for the next version of VFP. The August 2006 letter from Milind indicates the Fox Team is hard at work testing this compatibility. I am excited.

So now the SQL Server team decides it is in my clients’ best interest to use SQL Server Express? I might agree that this is a long term goal, but I have not certified my apps to do this, and in fact have struggled with SQL Server Express to be cooperative. I believe the “high point” reasons are not enough. Come on, Microsoft Update is a big reason? I take care of my clients: don’t be concerned with this Microsoft.

The support life cycle is a minor problem in my opinion, but significant enough to consider. But honestly, if the SQL Server team was forced to fix any Vista incompatibilities with a service pack then Microsoft would have to bump the life cycle expiration (if they follow with the same action they have done for other products). The customers (us and the clients we service) would be big winners.

I understand the benefits of moving on up (except for the seriously slow Management Studio Express applet), but there are cold realities in this world of business and it is not always the practical decision to “upgrade.”

I hope Microsoft understands why I will not be recommending Vista to my customers who need and demand MSDE. I guess I can continue to load it on a Windows XP Pro machine, or even the “clunky” Windows 2000 machine sitting in the closet for those clients who want to get Vista-ized on their new desktops. But the clients who use MSDE are usually in a small office and I will just have to convince them to keep an older OS around. Or maybe I will have to recommend another backend database (Whil and Ted and the rest of the MySQL proponents need not comment {g}).

This is just one bonehead decision, and one I hope we can rise up and influence to change. It has been done before so the trail has been blazed. Get those emails sent.

I will post my findings as I learn more.

*< RAS 9-Aug-2006, keep commented until this problem is debugged.
*< SET RANT OFF

Recently I blogged about data overflow not always happening with numeric data in Visual FoxPro. This was a frustrating problem to track down and yesterday I ran into another side effect of this problem when using TRANSFORM().

If you execute the following in the Command Window:

?TRANSFORM(6500.0000/0.00015)

You get 43333333.333333340 dumped on the VFP desktop.

If you try to store the result of 6500.0000/0.00015 into a N(12,5) column in a table you get 43333333.333 even though one could expect an overflow condition to occur. But as I relearned a couple of weeks ago, this is not the case.

I was debugging this same conversion I originally discovered this behavior with last time for a different overflow problem. In the text output file I was seeing the asterisks for the column storing the large number. In the intermediate and temporary DBF of the output I was seeing 43333333.333. This particular field is numeric in nature, but I pass it along to the receiving system as a character field per the spec. To convert the number I elected to use TRANSFORM() because they did not want any leading spaces or trailing zeros. The STR() function returns asterisks by default on the large number and requires me to specify the number of digits and the decimals to avoid the overflow representation. TRANSFORM() was not only easier and required less code, but also met the exact requirements for this situation.

But yesterday I was seeing stars. Stepping through the code I was observing the values without overflow until I hit the code to TRANSFORM() the column to a character representation. The result was the overflow asterisks. Argh.

The VFP Help file indicates their can be problems with large negative numbers not getting correctly transformed, but says nothing about large positive numbers. What is happening here is that VFP realizes the overflow condition in the table column and transforms it correctly to asterisk.

The problem was tricky to figure out and time consuming to track down, but thanks to the new CAST() function the solution was easy. I casted the field early in the conversion process to ensure the column is wide enough. The real kicker is the data was bad data (not that I am being judgemental {g}).

Yesterday was a pretty frustrating day all the way around, and this was just the frosting on the cake.

I just got off the phone with Bob Kocher who tirelessly works on the Southwest Fox Conference every year so we can all come to Arizona and have a ball, and learn about our favorite development tool – Visual FoxPro. Bob and Sharon put on a fantastic conference as many of you already know from reading my blog, other blogger’s posts, the UT Conference notes, and posts on the various Fox forums.

I am very passionate about going to conferences. I feel they are the single biggest way one can jump start a career advancement move. It is a way to rejuvenate your spirit and it brings fresh ideas instantly to your current projects. I feel everyone should attend at least one conference a year. I also feel Southwest Fox is a great conference!

I made Bob an offer this evening to back up my passion. This is the first time Therese and I have done this. I am proud to announce that White Light Computing is offering one scholarship to attend Southwest Fox 2006. This is a US$300.00 scholarship (half the current “tuition” to register for this year’s event in October).

So how do you qualify for this? Simple, register for Southwest Fox 2006 between now and August 15, 2006. White Light Computing and Southwest Fox will randomly draw one name from the list of attendees who register between August 1, 2006 (today) and August 15, 2006 after the registrations for those days are processed.

This means you have to pay for the conference now. White Light will send you a check in the mail (or to a PayPal account) for US$300.00 before the end of the month (definitely before your credit card bill is due).

Just a few rules and regulations and notes:

1. One name will be drawn.
2. The winner will be notified by a phone call from Rick Schummer.
3. The conference registration cannot be cancelled. You may send another developer/person to take your place, but the registration cannot be withdrawn.
4. You must be 18 years of age to be eligible.
5. Except where prohibited, participation in the scholarship constitutes winner’s consent to White Light Computing and Southwest Fox to use winner’s name, likeness, opinions, biographical information, hometown and state for promotional purposes in any media, worldwide, without further payment or consideration.
6. Registrations must be processed before midnight (Mountain Time) of August 15, 2006.
7. Each full conference registration is one entry in the drawing.
8. Odds of winning will depend on the number of registrations during the allowed period.
9. The judges rulings are final.
10. All tax considerations are the responsibility of the winner (consult your tax accountant for specific details).
11. No employees of White Light Computing, or any family members of the employees of White Light Computing are eligible.
12. Your mileage may vary, some settlement may occur during shipping.

Your boss is giving you a hard time? Let me know. I am willing to talk or email each and every one of them to help you get the okay to head to any FoxPro conference. I am totally serious. I have fought numerous times with my old bosses to get myself to a conference as well as teammates. I have a way of getting them to see the light!

White Light Computing is having a fantastic 2006! This is just a little way Therese and I are going to pay it forward!

So what are you waiting for, get moving, tell your Fox friends, get registered for the best conference of the year!

Good Luck!

Anyone who has worked with VFP data will tell you the DBF is a fixed record length and structures do not have variable length columns, even with the introduction of data types like VarChar. A VarChar is still a fixed length field in the table structure, but internally VFP treats it differently when you work with the data. If you design a numeric field 5,2 you have two significant digits, a decimal place, and two decimal places. Right??

Yes, but only most of the time. {g}

Today I ran into a situation that probably has happened to me only once before in more than 17 years of working with DBF files. I am working on a death march conversion. Part of the conversion does some mathematical calculations with cost. The columns we are fitting data into are N(5,2) so any information 99.99 or less should fit just fine. We ran the conversion and the client was complaining about data being off by 2 cents. We figured there was some rounding problem with the data. My co-worker on the project watched the process in the debugger. The data in the SCATTER NAME object had 78 cents (.78). The next line executed is an INSERT INTO … NAME and in the column was 80 cents (.80).

I was asked by my co-worker to confirm he was seeing what he thought crazy behavior in VFP. I asked him to BROWSE the table. Sure enough it was true. This is sort of what we saw in a column with two decimal places of precision.

89.54
98.29
05.78
147.8

If it was a rounding problem we would have seen several of the values round to the tenth. What caught my eye was the missing hundredths value in the last and current record. At worse I expected .80, but in this case we saw a three whole digits and one decimal place stuffed into a N(5,2) column.

Argh!

Visual FoxPro is dynamically interpreting the structure a different way. I think it should creat an overflow condition, which immediately sticks out like a sore thumb. Instead, we lost a couple of hours of time tracking this down.

It is not until I enter in a numeric field with more than 5 significant digits that I get “Numeric overflow. Data was lost” (Error 39). Interesting behavior for sure. I will not be reporting this as a bug because it is probably the way DBFs have worked since the beginning of time, but it sure can lead to some expensive debugging sessions if you are not aware of this behavior.

I wrote a short test program to see if this is an issue with a SCATTER NAME object vs. doing a standard APPEND BLANK and a REPLACE, but the behavior is the same.

Hopefully this blog entry will save you some time with this problem down the road.

(Somewhat off topic)

So where have I been? Vacationing – yep, a real vacation.

This year I decided I am going to vacation like normal people. You might not know this, but most people actually leave work behind when they go on vacation. So I did this over the last twelve days. Okay, not all twelve days, but for the majority of the twelve days. Heck I was disconnected from the Internet most of every day, and sometimes all day. One day when I reconnected I downloaded over 1000 messages.

Last fall while watching The Amazing Race (television show on CBS here in the USA) we saw some beautiful country when the teams visited Yellowstone. It reminded me how much I have wanted to visit this part of the country. I have wanted to go there ever since seeing my grandfather’s slides when I was a young boy. Last month I convinced the family to go. They voted unanimously for this. The other option I put on the table was to visit Louisiana/Mississippi where Katrina hit, and Houston, Texas (for Johnson Space Center). Seriously, I thought seeing the destruction first hand would be something good for us to experience.

At first Therese was a little nervous when I mentioned camping without electricity. Then it hit her, if Rick does not have electricity he has no way to run his computer once his battery runs out. She was sold.

We packed up the family (including my parents and nephew), connected up the camper, and headed west on a 1800 mile journey through Michigan, Indiana, Illinois, Wisconsin, Minnesota, North Dakota, Montana, and Wyoming. We also stopped by Idaho since it was still on my list of states to visit. I am now down to three (Alaska, Louisiana, and Oregon -I plan on visiting two of the three next year).

Yellowstone was spectacular! It surpassed all my expectations except for not experiencing an earthquake. We saw everything from Old Faithful to the Grand Canyon of Yellowstone. We went to the top if a 10, 000+ foot pass to get there, visited each and every entrance, and hiked though geyser infested areas. The Mud Pots were cool. The falls were spectacular. Revisiting the Continental Divide is always fun. The views just kept getting better all week. We saw a ton of Elk, dozens of Bison, and even on Grizzly Bear. The weather was spectacular.

We also visited the bridge where Drew and Brent died last September. It reminded me of how fragile our lives are. Being there also brought more questions to mind than answers on how such a tragedy could happen to these two.

After Yellowstone we visited the Grand Tetons. These are the mountains I have always wanted to visit. I have seen mountains in Colorado, Washington, California, and Tennessee, but these mountains are the best I have seen. Therese said these were the closest mountains she has seen to the Alps.

I took over 400 pictures and an hour of video.

On the way home we headed out the east entrance of Yellowstone and drove to Mt. Rushmore in South Dakota because my nephew has not been there. We also travelled through Iowa, Illinois to get back to Michigan.

We have decided Illinois is the state most unfriendly to travelers with RVs (sorry to all my friends there). They restricted us to 55mph when everyone else can drive 65mph including some trucks, they charge more than triple the tolls over the price of a car, and we had long traffic jams around Chicago. One toll booth was responsible for a 15 mile backup and then they charged us US$8 when cars were charged US$2. Appalling.

In Michigan you can travel on our roads for free. So come to Michigan where vacations are better for drivers of all types.

Wyoming is a fantastic state to visit. I have to rate it as one of my 5 favorite to visit. If you have not visited this part of the country, make sure to add it to the list of places to visit. You will not regret it.

So we drove over 4000 miles, added three new states to our list, stayed in some great hotels to and from Wyoming, and one really crappy Holiday Inn Express in Benton Harbor, Michigan (they stuck us in a smoking room when we reserved non-smoking, and told my parents to come get a plunger to unplug their toilet when they arrived to a plugged toilet). The hotel did not even appear to be near capacity, but they refused to put us in a better room.

Overall – one of the best vacations I have ever had. Now for the fun part, getting back to work. I guess it should not be too hard since I love what I do. Good thing, I have a lot to do! {bg}

I am a huge proponent of code walkthroughs. There are many benefits:

1. Cross training team members on development language
2. Cross training team members on business the software serves
3. Lower bug counts per release
4. Enforcement of coding standards
5. Flushing out potential inefficiencies in the code
6. Communication of development techniques

A little over a week ago I blogged about VFP Macro Substitution Techniques and I received a couple of private emails offering suggestions for the code examples I posted. One in particular was very helpful from Sergey Berezniker (major VFP guru, VFP MVP, and UT posting bot {g}). He introduced me to the GETPEM() function. So instead of this code:

lcPropertyValue = loPickedControl.&lcProperty;

I can use this code:

lcPropertyValue = GETPEM(loPickedControl, lcProperty)

As Sergey points out, the macro substitution code will be slower. Now I could argue the time savings will be immeasurable to the human mind, but the code is way more readable. This is what code walkthroughs are all about!

I have seen developers get very defensive on the constructive criticism side of the code walkthrough. I on the other hand welcome suggestions to make me a better developer. This morning I did a code walkthrough with my son who is subcontracting for me on a Web project. I know little-to-nothing about PHP, yet I was able to make some subtle suggestions to his code to make the application better.

It is important when doing code walkthroughs to not attack other developers. This is one of the simple golden rule things. Treat other developers with the same respect you want when you are the developer who is in the “hot seat.” I regularly and informally walk through code for other developers, especially those I am mentoring. You would be amazed how much I have learned by doing this.

So thanks again Sergey, and others who were kind enough to nudge me with suggestions. I am a better developer today because you took the time to do so.