I was reading James Whittaker’s posts on  Crowd Sourcing at http://blogs.msdn.com/james_whittaker/archive/2008/08/20/the-future-of-software-testing-part-1.aspx, where he sees the next big leap to be one where everyone who wants to can be a part of the Testing Solution.  While I’m not quite sure I agree with it, and some of the quotes reflect my own opinion, I can see where it can be of benefit as an additional avenue of allowing people time and a chance to test software and submit their issues to be fixed.  This is a good thing, most people, as he says, do find problems not always found in a test cycle either because of contraints, or some quirk of the User Environment that is not mimiced in the lab; getting those issues in early is a good thing.  However, I am a realist, and while its nice to get many of these bugs in I personally don’t see where the value is going to be if those bugs are not being resolved but end up in the bug bucket waiting to be looked at.  Though I am getting ahead of myself, there were a couple of points in this I thought was interesting.
 
The Cloud.  I’m still unclear on what this is, or whether its just an anticipation to what Cloud Computing is going to be, but I’m not expecting people to be sharing configurations and environments across the Net.  In a QA lab you can have all kinds of virtual environments, but they are not personalized, they tend to be representable of what Customers have, either by being proactive and knowing what Customers use, or by being reactive and adding in software or configurations that became known trouble spots.  I’ll share a specific browser configuration and plug-in configuration with someone, but I’ll be damned if I am going to sit there and share someone’s Hello Kitty theme with kitty cat icons made up of the heads of someone’s little babies.  I’m sure that works fine for some people, including some of my co-workers (not the Hello Kitty theme, but the cat picture backgrounds) but its not something I’d expect to see in my lab nor would I expect it to have issues with commercial software.
 
Bug Reporting.  As I said, just because Jim from South Carolina found a particular GUI issue with the tool bar, and it was confirmed with Vijay from Pune as well as Klaus from Dresden, doesn’t mean that bug is going to be fixed.  I’m all for finding as many bugs as possible before I release, but honestly, do I need 100+ minor defects or 250 Enhancement Requests sitting in my bug queue because the Crowd found them, and the serious issues they had were already reported and entered?  I don’t see what this gains me, other than more defects for triage that may or may not ever get fixed, but from a business standpoint I can say that we found them with the Crowd!  Serious defects should hopefully be caught prior to the release to the Crowd, or maybe specific configurations can be found, and I am all for getting as many of those as possible, but when I look at the numbers of Critical and high priority defects being added in, there are very few found later on; while I’d like those fixed is there any guarantee they will be?  I’m still waiting to see the results on that.
 
Isn’t this really Beta?  I can see this point, and I partially agree with it.  If you are working on an open source project you expect updates that may or may not be very well tested, with a Beta you are taking this knowing that the software may crash (and badly) in your environment because that is what it is.  So what is this?  Beta?  Well, I’d say not really since it seems like the releases are still very early in the cycle, unless the company has a long timeframe between a release build and eventually getting it out.  So where does the testing stop?  Does the release mean everything found is put on hold and we now wait for the next release, especially since Customers now have a copy we can be getting reports from them.  I’m waiting to see more on this, but I am trying to keep an open mind.
 
Talent Pool.  Who are the people who are joining?  Are these people new to the field and they want more experience?  Are they bored?  What is their experience?  As someone who has trained people off and on I can tell you that a bug report from someone who is just learning compared to someone who has been doing this for a long time is very different.  I’m not sure who is signing up here, the money doesn’t seem like much, and maybe I am not their talent pool, but when I get out of work the last thing I want to be doing is testing some other kind of software, heck I don’t like to practice programming much outside of work.  I’m not the target audience for the community, but I’m curious as to who is.
 
Unlike Mr. Whittaker I don’t know if this is the next logical step, but I like to take a long view and a higher view of testing as a whole, some things get tried, some change and some just end when its seen they don’t lead anywhere.  I don’t know where uTest and the Crowdsourcing is going to go, but I am curious to watch and see.

Tweet This Post

After the Development team came back from the latest Microsoft PlugFest there was a new version of the IOStress tool, which has a better structure than the last one without all the useless command scripts in the root directory.  It seems to run better, and has picked up on configuration I had set for the previous one, as I was able to just put the directory down and run it with fewer errors than I had seen in the last version.  Running with the Verifier and our Driver still causes a crash, but that is expected for the way the Assertions were being used in 2008, supposedly in Windows 2003 this was more forgiving, but 2008 doesn’t like it.  I’ve crashed the 2008 box 3 times since installing the new version, go me!
 
I need to get a listing of the tests, and what they do, but for now I have a nice basic structure that runs on our driver and doesn’t crash the box, always a good thing.  The one thing I don’t like is the reboots the IOStress program does when setting up certain tests and when completing them – since I run this in a remote desktop window and do other things – the test goes away when I don’t notice that the remote desktop is gone.  Part of the problem with the reboot is that the test results window disappears quickly after the reboot, I may have to look at getting the mail piece working so I can get the results in email, they may be formatted better.
 
Undocumented and unofficial tools are so much fun to work with….right.

Tweet This Post

In my current position I test a filter driver, basically a driver placed into the stack on multiple platforms that gathers events on the system and passes them up to a Java agent to send to a central server for review.  Some events are filtered out, and some are not, but default and this is done to keep the messaging down to a level where the driver is not stressed out, but sometimes its good to know how the driver operates under stress.  There are a few scripts we use to load the system with events of all kinds, but Windows provides a suite of tests in something it calls IOStress, and is given out at Plugfest every year.  I have spent some time setting it up for testing in my environment and this is what I have had to do so far to get it to work:

The IO Stress program needs a net share set up for ntiosrv that points to the iostress folder and supporting files
Copy the iostress folder from <server>\Software\Microsoft\iostress to a local drive (say D:\) and rename the io.stress60 (plug-iosrv) folder to iostress
Create the share for ntiosrvnet share ntiosrv=d:\iostress /GRANT:Administrator,FULL /REMARK:”IOStress Share”

This will set up a share to the D:\iostress directory, change to that location and run – iostress.cmd /ignoredebugger
The I/O Stress program main window may be minimized, select it
In the Registration and Verifier screen enter the driver to be tested

Select Low Resources Simulation
Select I/O Verification Level 1
In the Registration and Verifier screen, if run with No Debug, there will be a note regarding No Debugger Selected, this is due to the Email and Contact information being empty. Enter values, any will do, into the three fields.
Select the drives needed to be tested, all available drives on the machine will appear in the Run Information tab, on this page as well the Test Time can be entered in the Specify Number of hours to run field.
The Stress Tests tab allows selection of specific tests, a slight overview of the tests is included with the zip file that contains the tests, it gives some information on the basic tests.
If the system reboots you must log in to the machine, the IO Stress kit likes the Administrator account to have a blank password, but that is insecure
Still, even with all this I have encountered a few problems when running the tests:

There is a complaint about a registry value not discovered, I have yet to find documentation on what that value is, but it seems to still run
Do not run with the Verifier, this seems to cause crashes, possibly because the IOStress program runs with the driver in the Verifier and having them in twice causes conflicts
When setting up the environment it claims some network drives are not found, I am not sure if its the ntiosrv share or not, I can never seem to find it in the net view list, but without the share being available the entire test suite will not even start.
I don’t mind Windows tests, and undocumented Microsoft tools are such a joy, but it does some checking and has allowed me to verify some issues and find new ones especially on Windows 2008 which is a new platform for my group and we don’t have a complete set of tools for that platform yet.  If I get these issues resolved, I’ll note the fixes.

Tweet This Post

Quickbooks Meetups > Bay Area Quickbooks and Business Coaches

Presenter 1: Patrick Schwerdtfeger
Tactical Execution
Leveraging Today’s Social Internet
– Learn simple strategies to develop a massive online audience with little or no money.

Patrick is the host of one of the nation’s largest entrepreneur meetup groups and where I learned the meetup ropes. Glad to be able to get someone like Patrick for this meeting.

Presenter 2: Joe Lindsey
The Turning Point/webKPI

Joe has assisted numerous start-ups with developing and implementing proven sales strategies.

Oakland, CA 94612 – USA

Wednesday, March 18 at 6:30 PM

Attending: 8

Details: http://www.meetup.com/QuickBook-Business-Coach/calendar/9351168/

]]>

Tweet This Post

I’ve been meaning to write this one up for months now, ever since I originally posted the sexy forms in rails. The form builder I talk about in that post makes it easy to put a submit and cancel button on a the form, and a few people have asked how to handle the cancel action. Some have suggested using javascript to redirect back using an onclick, ick! What happens if the user doesn’t have javascript support? Sure you might try and justify it by saying, “Well, how many users don’t have javascript these days?” but there is absolutely no reason why this has to use javascript, so lets make it work for everybody.

It’s actually much easier than you’d think, within application.rb I’ve got the following:

class ApplicationController < ActionController::Base
  before_filter :catch_cancel, :update => [:create, :update, :destroy]
  after_filter :set_referrer,  nly => [:index, :show]

  private
    def set_referrer
      session[:referrer] = url_for(params)
    end

    def catch_cancel
      redirect_to session[:referrer] if params[:commit] == "Cancel"
    end
end

The logic being, that if we’ve got a fully RESTful architecture the only pages we’ll ever want to go “back” will be the index or show actions on each controller. So we cheekily call urlfor in an after filter with the current params to get the path of the page we’ve just display the user, and stash it in the session. Theoretically it may be possible to use ENV[“HTTPREFERER”] instead but it’s not guaranteed, this approach however ensures we keep track of the page to go back to explicitly ourselves.

Then, to piece it all together there’s a before filter checking if the user has clicked the submit button labeled with “Cancel”. You’ll need to change that equality test dependent on what text you put on your button. You may also need to move the before filter into the controllers that require them, depending on the order it needs to run with other filters you have.

Hope that helps.

Tweet This Post

Online Community Report Meetups > The San Francisco Online Community Report Meetup Group

The fourth Wed. of the month is the free monthly gathering of online community managers, enthusiasts, and innovators to meet and discuss tools and strategies for building and managing effective communities.

We had a great turnout for the Jan. MeetUp and hope to repeat the same in Feb.! Music and/or community fans alike, this one is right up your ally! Our guests this month are Amy Miller and Gabe Benveniste of SonicLiving, a concert tracking tool and social network that syncs with iTunes, Pandora, and Last.fm to find local concerts based on a user’s personal taste. They’ll give a short presentation and demonstration of SonicLiving which will frame our follow-up discussion, with time for networking and refreshments before and after.

Hope to see you there!

Speaker Bios: Gabe founded SonicLiving at the end of 2006. Prior to that he was at Pixar for over 6 years where he created collaborative websites that streamlined the workflow of many Pixar employees. Gabe has spoken on behalf of Pixar at WWDC and O’Reilly’s OSX Conference. He finished sixth at the 2003 World Beard and Mustache Championship in the first ever sideburn category. His work has been featured in Playboy, MTV and Wired.

Amy is SonicLiving’s Community Manager. After working in non-profit community management, and being an obsessed show-goer, she found SonicLiving in 2006 and joined the team. You can say hi to her at a bunch of local shows and on Twitter.

San Francisco, CA 94107 – USA

Wednesday, February 25 at 7:00 PM

Attending: 30

Details: http://ocr.meetup.com/135/calendar/9625331/

]]>

Tweet This Post