I began reviewing water blasters, primarily Super Soakers, back in 1998 (over 17 years ago) to go with the launching of Aquatechnology. Of course, I never would have guessed then that I’d still be doing reviews 17 years on (but that’s another story). As Aquatechnology grew and I amassed more and more water guns to test, more people started visiting my site. As Aquatechnology evolved into iSoaker.com in 2000, the ways I measured and reported on how particular water blaster models behaved also changed. To think that there are now over 300 reviews on iSoaker.com
A Brief Historical Perspective
To look back at things, let’s look at the same water blaster model reviewed from three different time periods, namely the Super Soaker CPS 1500.
In the early days, I measured the following:
- Dimensions (Length, Width, Height)
- Weight (not available yet)
- Tank Capacity (now known as Reservoir Volume)
- Firing Chamber Capacity
- Pump Volume
- Range (level and angled)
- Penetration Value – how many layers of towel a stream could get through in a short burst
By 2001, the following stats were provided:
- Dimensions (Length, Width, Height)
- Weight
- Reservoir Volume
- Firing Chamber Volume
- Pump Volume
- Range (level and angled)
- Output
- Average Shot Time
- iSoaker.com Ratings: Power; Weight; Capacity; Overall
Today, the following information is summarized:
- Manufacturer
- Class
- Item Number
- Copyright Date / Release Date
- Known Patents
- Availability
- Weight
- Reservoir Volume
- Firing/Pressure Chamber Volume
- Pump Volume
- Known Version Colours: Body; Reservoir; Pressure Chamber; Detailing; Notes
- Number of Nozzles
- Range (level and 45 degrees)
- Output
- iSoaker Output Rating
- iSoaker Power Rating
- Shot Time
- iSoaker.com Ratings: Power; Range; Encumbrance; Ergonomics; Capacity; Overall
As can be seen, the number of details reported on any one water blaster model has significantly increased since the beginning.
But, so what? Just because more statistics are being reported on doesn’t mean there’s an improvement on how they are being measured!
Measurement and Statistics
For an overview of how water gun statistics are measured, see: Statistics – iSoaker.com.
There are two main attributes to any measurement:
- precision – repeatability/reproducibility of a measurement
- accuracy – closeness to a measurements’ true value
See: Wikipedia: Accuracy and Precision
Some of the things measured for a water blaster on iSoaker.com are fairly straight-forward. Water blaster weight (dry), dimensions (height, width, length), and number of nozzles are all fairly easy to determine and few question the numbers reported.
However, some things are more difficult to measure accurately, partly from the actual challenge in measuring them and partly from the definition and/or other variables involved when taking a measurement.
Range measurements are a perfect example of a problematic statistic to report on. When doing a range measurement, one needs to be able to determine how far a stream reaches. The problem with this simple statement are many-fold. Streams are composed of liquid and most tend to break up during flight, resulting in an area hit as opposed to a more confined spot. As such, when reporting range, should one report the farthest drop, the closest drop, the mid-point, or the median of the splotch? How far a stream fires is also a function of pressure available, duration of a shot, stream thickness, and initial height off the ground. Is one also reporting range from the User or range from the nozzle (the latter matters since some water blaster models are longer and can add 30-60cm (1′-2′) to a range stat). For a pressurized-reservoir-based water blaster, the amount the reservoir is filled will affect how evenly the pressure is maintained throughout the shot. For a separate-firing-chamber water blaster, things are also affected to some degree based on whether the chamber was pre-pressurized with air before or not. For bladder-based blasters, voiding air in the chamber should be done to make the shots more consistent. The problem is that it is unclear when someone reports a range on exactly what they did in preparation for the range test. Pressurizing aside, stream lamination which is greatly affected by the quality of the nozzle greatly influences range. Then, there’s the wind element. While we try to do range testing on calmer wind days, there remains the odd breeze/gust that may occur during a particular shot. To reduce such effects, we do multiple shots to check for consistency before recording a range.
However, even if one could control all the above variables, the variability within a particular water blaster model is a definite unknown. Though anecdotal, I recall going water blaster shopping back in 1999, trying to pick up a bunch of Super Soaker SC 500s for a work-related picnic. I was smart enough to bring a large bottle of water with me for testing since I had had issues before with another water blaster and figured, if I test it just outside the store, they really cannot complain I had broken it at home. The first one I picked up leaked from the pump when pumping so I returned it and opened another to find it also leaked. After eventually testing four, I gave up and informed them their batch must have been bad since I already had an SC 500 and it should not leak like that. The store representative was admittedly frustrated, but not so much with me, but more of with having to deal with an apparent batch of problematic water blasters. Extrapolating from the experience, had I opted to pick up a few SC 500s and subsequently do a review on them, I probably would have concluded that SC 500s were prone to leaking from their pumps and range measurements would have undoubtedly suffered. I’ve used 10+ SC 500s since then, none of which leaked from their pump. Therefore, what is average? Should I include ones that leak or not? But what if it turns out most people found that their SC 500s leaked? Wouldn’t that make the one’s that didn’t leak unusual, there’s shouldn’t be counted?
The problem with range reporting is that if a range reported isn’t within a few feet of where another User finds their water blaster shoots, people complain. Even if one could test many water blasters of the same model and determine an average range of X, someone, somewhere, will have a rarer version that performs generally better or worse. If that particular batch was better for some reason, their friends may also have these better performing versions and claim that the “true” range should be Y. Furthermore, they would feel their number is better since they tested “several”. To get more a more accurate average range, due to all the variables involved, it wouldn’t take tens or even hundreds of tests; the number is in the thousands or higher. As that is both hugely expensive and time-consuming, we have opted to stick with reporting the range of our particular version tested and leave it to others to report on their range measurements as well.
Range is not the only problematic statistic. Reservoir volume is another. “But, hold on, why is that a problem?” some may ask. For blasters with a separate pressure chamber, reservoir volume is simple to measure – simply fill to the top, then pour out into a measuring device. However, for a pressurized reservoir volume, the number reported is less clear. Should the reservoir volume be reported as the total possible amount of water the reservoir could hold, even though if 100% filled, the water blaster would NOT be able to shoot properly? Or should an optimal 2/3 or 3/4 fill volume be reported? For those familiar with the technology, they may prefer to know the total internal volume of the reservoir since they have their own, preferred fill level to use. However, for those less familiar, they would consider reporting on full volume an over-estimation of how much water one should actually fill a particular water blaster with.
Unable to easily address the above concerns, all iSoaker.com Review Pages simply state in their notes: “Statistics measured are from products and water blasters tested here at iSoaker.com; individual water blaster performance may vary.” Some may say this is a “cop out” compared to trying to measure more water blasters in hopes to get a more accurate number. However, in light of the fact that it would take tens to hundreds of water blasters sources from various locations, each tested multiple times, to really get closer to a more accurate number, that simply is not possible, especially for a lone individual to do. Now, if there are tens or hundreds of other people willing to do their own measurements and share them with iSoaker.com, that would be the most realistic way to try to obtain more accurate statistics.
In the end, yes, I cannot confirm the accuracy of all the statistics presented on iSoaker.com. I can comment on the precision of measurements and, of course, provide insight regarding how various measurements were obtained. Of course, I want to present more accurate statistics whenever possible, but that really requires obtaining a lot more measurements more a lot more sources than I can do on my own. While it would probably require hundreds of measurements to be analyzed, I could argue that when the number of measurements get above 50, the result should be closer to the true average value. That said, perhaps I should begin compiling at least range data from various sources and when a particular water blaster model ends up having at least 50 unique range measurements, I’ll report that number in addition to the value measured here. Always take any statistic anyone reports with a grain-o-salt since statistics are really only as good as how they were obtained. Anyone got stats to share?