Tannhauser Posted November 23, 2003 Share Posted November 23, 2003 Be warned, this will be lengthy. Just some thoughts I thought I’d share on a wet Sunday. Also, I want to make clear that I’m not dissing anyone on here – various traders and experts on here have helped me a lot – it’s just a kind of theoretical monologue on the problems of getting evidence. Years ago, when I was into bodybuilding, I must have bought every nutritional supplement known to man: protein powders, vitamins, amino acids, etc, etc. The problem, as a punter, was to be able to figure out what was effective and what wasn’t. Now that I’ve been sucked into the world of performance mods, I find I’m facing the same problem – the difference being that this time round, the products on offer cost many times as much than in bodybuilding. There’s been debate on this BBS and other forums on loads of mods. For example, the effectiveness of after-market induction systems, water injection, fuel regulators, hybrid turbos and so on. Often, one person swears by a product whereas another says that it represents very modest benefits (or none at all). What’s a punter to do before shelling out his cash? This got me thinking about the types of evidence that are available about performance mods (and shortcomings with it): 1) Someone who fitted the mod says the car felt faster. There are two problems with this. Firstly, human memory is not fantastically accurate. There may have been a considerable gap between driving the car before and after the mod is fitted – and we know how fast a supe feels after you’ve had any break from driving it. Secondly, there is the placebo effect. If you’re expecting the car to go faster, it may well seem to go faster, regardless of its actual performance. The power of expectancy on perception has been reliably demonstrated in research studies hundreds of times. 2) Someone else has a car with the mod and it feels faster than yours. If you were a passenger, a sensible comparison might not be accurate: it always feels faster from the passenger seat. But even if you’ve driven both cars, can you really attribute the difference to that mod? We know that completely stock cars vary widely in performance and this effect must increase with age. And there’s still the ‘expectancy effect’. 3) Someone who fitted the mod did some before-and-after road tests. As a source of evidence, this is much better than the above. But, of course, there are many variables apart from the mods that might affect the speeds obtained: road conditions, temperature and so on. Then there’s driver effects, too. Ideally, you would want the experiment conducted ‘blind’ i.e. you didn’t know whether or not the mod had been fitted at each test – but the chances of someone actually doing this are next to zero. Incidentally, IanC road-tested his HKS cams and restrictor ring and published the results on here. That was fantastic, because he even documented the differences between each run (e.g. window open or closed!) but I can't find any other examples of anyone doing this. 4) Someone’s track times improved following the modifications. Same weaknesses as above. 5) A guru recommends this mod. It sounds promising to rely on someone who has theoretical knowledge and real world experience that are miles beyond your own. However, as in all areas, experts often disagree on the value of a particular mod. Also, in addition to some genuinely knowledgeable people, there are no doubt others who understand less than they think they do. In the country of the blind, it might be hard to tell the difference between a one-eyed and two-eyed king. Take Mycroft, for example – I have read many of his threads on different boards. I simply don’t have the knowledge to know whether his controversial assertions are cobblers or not. All I can do is rely on the consensus of other gurus! 6) A mod manufacturer says it works, or provides test data. Unless the tests are truly independent, which is unlikely, then caveat emptor. 7) A trader recommends this mod. I’m not impugning the integrity of any trader on here –I’m not implying that they sell products they don’t believe in at all. In fact, compared to many other consumers, we have an advantage in that the people selling the products have a passion for their area. But ultimately, few traders have the resources to ‘properly’ test the products they sell. They rely on consensus of opinion and customer feedback – both of these are usually based on the types of evidence given above. 8) A dyno comparison showed cars with this mod to generate more torque or hp. This is what started me thinking about this subject. I tried to compare the results from the Thor dyno day to see if I could work out the ‘value added’ for different mods. The problem is that there are too many variables (other than a particular mod) that might affect performance: transmission type, level of boost, UK or J-spec, other mods. Even if you can find similar cars too match, you still have the problem of variations in stock power from vehicle to vehicle. So although all of the above sources of evidence might be useful in coming to a decision about a mod, there are smaller or larger problems with all of them. It’s a pity that the technology surges onwards, the availability and diversity of products increases, but the testing lags behind. One way of getting a more valid source of evidence might be as follows: a) Vehicle taken to dyno with proven reputation that can demonstrate low variance between readings (i.e. reliable measuring method). b) Car is ‘prepared’ away from dyno guy. So, the dyno guy is working ‘blind’ – he doesn’t know if car has had modification or not. This might seem unnecessary, but if you want a truly ‘scientific’ test, it’s almost obligatory. c) Car is tested several times and mean figures calculated. d) Car is removed and alteration (adding/subtracting mod performed). e)Car is re-tested, attempting to keep the values as close as possible to original runs. Trouble is, this is an expensive, time-consuming process. I have seen some dyno comparisons carried out on the US boards, but guess who did them? The makers of the parts. So there is the possibility that knowingly or unknowingly, they 'helped the results along'. Step forward any independently wealthy individuals who can hire a mechanic and a dyno and buy some bits to test them out……wot no takers?? In retrospect, I wish I had taken some before and after road test results when I had some mods done to my car. Although, as I say, the evidence isn't perfect, it's probably the best we can get in the real world. OK anyone still awake? Regards, Cliff Quote Link to comment Share on other sites More sharing options...
Supra Pilot Posted November 23, 2003 Share Posted November 23, 2003 Quote Link to comment Share on other sites More sharing options...
Nick_TC Posted November 23, 2003 Share Posted November 23, 2003 Cheers Cliff, that's just what i needed to get me set for my afternoon nap But seriously, some very good points raised. Quote Link to comment Share on other sites More sharing options...
Dragonball Posted November 23, 2003 Share Posted November 23, 2003 I suppose all the sheep are in for the night up in Wiltshire? Mods DO make you go faster - the only point that I can make out is that you would like to have an independant test on each modification to see what the real gains are? Pete @ THOR was going to put up some BPU and BPU+ info from the runs on his machine - must check to see how far he got Quote Link to comment Share on other sites More sharing options...
Gordon F Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Paul E Mods DO make you go faster Yeah, but dont forget that there is always a trade off when you 'modify' something. Could be a lack of driveability, reliability, increased mpg, anything like that. BTW - mine is totally stock and I have seen 170+. How much faster do you want to go? Quote Link to comment Share on other sites More sharing options...
dude Posted November 23, 2003 Share Posted November 23, 2003 Best bet is drive someones car with the sort of mods you are thinking of doing and make your own mind up , see how the car feels to you youll soon know if its faster , you know what i mean , if it looks like a duck and smells like a duck and sounds like a duck its usually a duck!!!! Dude:flame Dev Quote Link to comment Share on other sites More sharing options...
Tannhauser Posted November 23, 2003 Author Share Posted November 23, 2003 Hi Paul, I'm not disputing that mods (collectively) make you go faster at all. However, there has been debate on here about how truly effective SOME mods are. I'm saying that when it comes to trying to make decisions about whether something offers good bang-for-buck or not, we should be cautious with the evidence available at the moment. My other points are: 1) It's a pity the testing science lags behind the technology 2) Dyno tests of the same car offer the best way forward, but it's quite an investment of effort and money to get it right 3) Failing that, more road tests like IanC carried out would really help others. BTW saw your car yesterday, never thought of white as a good colour, but yours looks severely cool. Also had a ride in dude's and was highly impressed compared to mine...which also got me thinking about how hard it is to make truly fair comparisons. Regards, Cliff Quote Link to comment Share on other sites More sharing options...
dude Posted November 23, 2003 Share Posted November 23, 2003 Dont worry Cliff the finger marks came out of the dash Dude:flame Dev Quote Link to comment Share on other sites More sharing options...
Dragonball Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by dude , if it looks like a duck and smells like a duck and sounds like a duck its usually a duck!!!! Dude:flame Dev Haven't got a clue what you're on about - but it made ME laugh! Couldn't find this mod on our website m8 - this something to do with going single? Quote Link to comment Share on other sites More sharing options...
Gaz Walker Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Paul E Haven't got a clue what you're on about - but it made ME laugh! Paul, I thought you would have got that, being OLD and all Gaz. PS Nice quote Dude Quote Link to comment Share on other sites More sharing options...
dude Posted November 23, 2003 Share Posted November 23, 2003 , if it looks like a duck and smells like a duck and sounds like a duck its usually a duck!!!! talking of duck its what youll need to do if i find 1 more crap mod on your car Dude:flame Dev Quote Link to comment Share on other sites More sharing options...
Roy Posted November 23, 2003 Share Posted November 23, 2003 talking of duck its what youll need to do if i find 1 more crap mod on your car eh ... you gonna be a while then dude ? Quote Link to comment Share on other sites More sharing options...
Dragonball Posted November 23, 2003 Share Posted November 23, 2003 Agree m8 - hopefully Mr Wonderful 'he's with a customer' aka 'will call you back in 2005' Nice person will sort out all the electrics for me Matt - see you tomorrow morning!! Quote Link to comment Share on other sites More sharing options...
Matt Harwood Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Paul E Agree m8 - hopefully Mr Wonderful 'he's with a customer' aka 'will call you back in 2005' Nice person will sort out all the electrics for me Matt - see you tomorrow morning!! I want a Duck mod! And strange though it may seem, I do have other customers! And I've already said, I don't do call backs! Quote Link to comment Share on other sites More sharing options...
dude Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Matt Harwood I want a Duck mod! And strange though it may seem, I do have other customers! And I've already said, I don't do call backs! Matt dont you check yer phone , phone me you ignorant fecker !! Dude:flame Dev Quote Link to comment Share on other sites More sharing options...
Need4Speed Posted November 23, 2003 Share Posted November 23, 2003 Trouble is, this is an expensive, time-consuming process. There you have it in a nutshell. Quote Link to comment Share on other sites More sharing options...
GESIT Posted November 23, 2003 Share Posted November 23, 2003 Cliff, As a newbie to this game, i really hear where you are coming from. I do not really know who to listen to, what to buy....basically all your points mate:( However, for me i am finding a wealth of knowledge from all the guys and gals on here and the Traders are good too. I may be the man for your 'Experiment' though. I am still getting that funny whining sound from the car at low revs and was planning to do the following: 1) Take Stock J-Spec TT to THOR and get them to try and fix the issue. 2) While they do that, get a DYNO run. 3) Get my new Exhaust, Spark Plugs,Boost Controller, FCD, Air filter. 4) Then go back to THOR and do a re-run on the DYNO. It would be great to see the results after each mod had been applied in turn, but that would be very costly i think. The results may be interesting though Later G Quote Link to comment Share on other sites More sharing options...
Wez Posted November 23, 2003 Share Posted November 23, 2003 I originally planned to do dyno runs between mods but then i started modding and still haven`t been anywhere near a dyno yet. Although hoping to get on a dyno soon to tweak the AEM some more. Quote Link to comment Share on other sites More sharing options...
Ian C Posted November 23, 2003 Share Posted November 23, 2003 I'd like to see a test on FSE powerboost valves Glad to see the info was appreciated Tan -Ian Quote Link to comment Share on other sites More sharing options...
qaisar Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by GESIT Cliff, As a newbie to this game, i really hear where you are coming from. I do not really know who to listen to, what to buy....basically all your points mate:( However, for me i am finding a wealth of knowledge from all the guys and gals on here and the Traders are good too. I may be the man for your 'Experiment' though. I am still getting that funny whining sound from the car at low revs and was planning to do the following: 1) Take Stock J-Spec TT to THOR and get them to try and fix the issue. 2) While they do that, get a DYNO run. 3) Get my new Exhaust, Spark Plugs,Boost Controller, FCD, Air filter. 4) Then go back to THOR and do a re-run on the DYNO. It would be great to see the results after each mod had been applied in turn, but that would be very costly i think. The results may be interesting though Later G well i think what Cliff was saying is feedback at each stage ie for each mod. now obviously that would mean doing a dyno run after fitting each part which would work out expensive for one person and which is why it probably hasnt been done although perhaps one of the traders could do it alternatively how about we see if Pete can sort something out in terms of price for repeat dynos and everyone chips in a few quid to cover the dyno time? obviously we aint gonna buy ur equipment but i presume u r going bpu route so it would be valuable information for the bbs Quote Link to comment Share on other sites More sharing options...
Tannhauser Posted November 23, 2003 Author Share Posted November 23, 2003 I'm surprised that one of the modding magazines, like Japanese Performance, hasn't done some independent dyno tests on different bits of kit. They would be ideally placed as they have the financial resources and, at least on the face of it, a more independent stance. Of course, if they tested a mod and it wasn't any good, companies selling that kit through their mag might not be happy.... Traders are mostly selling someone else's stuff, so IMO asking them to contribute towards testing is hardly fair. It's like asking your local grocers to pay for tests on Heinz baked beans. Group-financed testing? Interesting, but I'm not sure how many would be interested. I might be wrong, but I think most are happy with the evidence we currently have (That sounds snobby but it's not intended to be). I have a background in science, and I think it might be this that makes me hanker after double blind studies with quantitative results, yadda yadda yadda. Cliff Quote Link to comment Share on other sites More sharing options...
Wez Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Ian C I'd like to see a test on FSE powerboost valves Glad to see the info was appreciated Tan -Ian Why an FSE Ian? Wouldn`t thought an FSE would increase power if anything you may decrease power if it is not setup properly. Quote Link to comment Share on other sites More sharing options...
qaisar Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Wez Why an FSE Ian? Wouldn`t thought an FSE would increase power if anything you may decrease power if it is not setup properly. but wouldnt it be possible to see how much fuel it helped to flow? Quote Link to comment Share on other sites More sharing options...
dude Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Ian C I'd like to see a test on FSE powerboost valves Glad to see the info was appreciated Tan -Ian That'd be like testing say Boost Controllers,you could see what they could do but not how well they can do it Dude:flame Dev Quote Link to comment Share on other sites More sharing options...
dude Posted November 23, 2003 Share Posted November 23, 2003 Originally posted by Ian C I'd like to see a test on FSE powerboost valves Glad to see the info was appreciated Tan -Ian Id like to see a test on the FSE as well so we can see how alltering the pressure effects fueling etc (its easy to do when you have an FSE) Dude:flame Dev Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.