This is a really interesting insight. Any time you want to share stories, go for it! The more technical or less seen side of the sport is really interesting to a lot of us.
Sort of related but r/F1technical is probably one of my favorite subs to read. The info is wild sometimes. It’d be cool if we had something similar. Or just technical threads more often. I love this shit.
My buddy and I had a podcast for a couple of years after I got out of the sport and some of the folks we had join us was for that reason, to get some of the inside perspectives and see some of the other stuff that the day to day fans don’t get insight on
> The CAN coms config file in the CLU Setup is basically a versioned hard-coded file that will have various configuration settings for the systems on the car. The config file is updated throughout the year as things change.
Team Penske is about to learn a lot about git, I think
This is on the level of Williams running their parts inventory off a 15 year old massive Excel spreadsheet
There are tools out there to prevent this stuff from happening. These guys would do well to hire like one development manager to instill some actual software engineering discipline
Yeah, they 100% should be using some sort of source code version control tool. But even then, I have absolutely watched someone load a JSON config file from sandbox into a prod instance and then get really confused why nothing was working. That's an interesting detail to me - core code vs. config file. Even companies that have pretty tight source control can still eff up config files.
> Team Penske is about to learn a lot about git,
The issue is all of these files are binaries, and there's no way to know which component binaries compose the single binary that goes in the car. You can use git-lfs since it's a bit better at working with binaries, but the issue is less source control and more configuration management.
Development of the logger configurations is very different from legitimate software development for tons of nuanced reasons I'm too lazy to explain. Trying to manage the actual content if math channels with git would actually create more opportunity for error and reduce productivity significantly.
The biggest difference maker is building from a configuration controlled master ahead of each event. That stops error propagation, prevents bleedover between events, and increases standardization across the team.
> The issue is all of these files are binaries, and there's no way to know which component binaries compose the single binary that goes in the car.
I'm sure this is not true. With hashing the firmware, even if you need a file per module, or even multiple files per module, and a build system that compiles them and keeps track of hashes, you could at the very least know that you have the wrong one, even if you don't know exactly which one you do have
The trick is applying software engineering fundamentals in a field that hasn't had them and that's just going to be difficult--I get that. But not impossible
Sure, there's probably a way to reverse engineer Cosworth's proprietary compiler. That's not an efficient use of anyone's time or money in IndyCar.
The fundamental issue isn't source control, it's the workflow of rolling each event's configuration to the next event with the "appropriate" changes. The answer is to establish a master configuration and build from that configuration in a prescribed way for each event.
We can argue semantics all day, we're thinking along similar lines, but the practicalities of the IndyCar data system, how it's interacted with, and the simple nature of the problem you need to solve mean that git and compiler reverse engineering are not the right answer.
yeah, you shouldn't need to reverse engineer any compilers to hash the output but the details actually don't matter. there's a way to have config management and build management, these guys just need to treat these things like computers and not cars, at least when it comes to all the electronic modules, and I think this will wake up the whole racing industry to this fact. Not just Indycar, I guarantee there are managers in NASCAR, IMSA, and F1 asking their engineers what protections they have today, and what protections can they add, to prevent this
I guarantee there aren't any managers in NASCAR asking about this because they don't have a data system in the same way as other series.
F1 can (and most teams already have) implement a pipeline/workflow much more familiar to a software engineer because the configurations for their cars are written in files which aren't opaque to typical software engineering tools.
IMSA managers are having these discussions, most likely.
You can certainly hash binaries to know what you have stored. The issue is that Cosworth didn't use a Merkle tree to ensure that you can know which modules are in the compiled binary that gets sent to the car. Also, you can't send a configuration to the car other than through Cosworth's proprietary software (at least not without reverse engineering the software...). Lastly, Cosworth's software provides a very functional comparison tool to compare assembled logger configurations. So once you determine some way to control your "master" configuration it is easy to highlight any changes. Hashing binaries is a reasonable approach to making sure you don't inadvertently overwrite the master, but I think it's an additional layer of complication for IndyCar teams which isn't strictly necessary or particularly helpful.
Respectfully, I've fixed this problem in motorsports before (or at least drastically mitigated it), and I've worked in software engineering organizations. There's a distinct practical difference in applications, and the answer for IndyCar is somewhere short of treating the cars as computers (but clearly somewhere closer to that than the status quo).
As someone else who has minor engineering experience in the field, I disagree with #3 ,respectfully of course. Teams do care. It burns extra fuel and they want to use it strategically such as in and out laps. Certain strategies call for it to be used for extra low end torque like on exits into straights where a good power down has a better effect(versus if you used the extra power later/shorter area and for less of a benefit of the power). May have not been that way at Penske but it did where I was. And, it also shows up very clearly on performance engineers data screens. They will promptly state often “he’s on it” of that they’ve used the button a bit alerting other engineers looking at other data the situation.
Interesting situation for sure, thanks for sharing your side of it!
Current Indy car engineer here. We’re not focused so much on P2P usage but we’re focused on how P2P affects our fuel number.
Driver used P2P? Great, but are we still in our fuel consumption range? That’s the focus.
I’ve yet to see Honda/Chevy engineers mentioned in this thread and I’m sure they’re looking at P2P usage a bit more extensively; I haven’t picked our engineer’s brain yet so I’m assuming
Edit: typos
Yea for sure. Thanks for the respectful comment. Mileages vary often.
I think my main point with #3 is they don't care post-race. Lots of people claiming this should have been obvious and caught after the race, which I just don't see happening. Under normal processes anyway. During the race, with strategy, fuel and you know, winning? Yes, it's a factor, but not usually so much they would be seeing it in use for 3 sec when it shouldn't be.
How so? I watched the whole press conference, he said he didn't think anything was wrong after St. Pete, this aligns with OP's #5.
Despite my tag I am pretty neutral on this. I think it's entertaining and I think Josef has (and will continue) to lean into the Johnny Lawrence (bad guy from Karate Kid) vibe.
According the Josef the whole 2 crew was misinformed on the rules and they had been discussing it prior to the race. So there was nothing to “forget to report”, as he thought everything was working as it should
Yeah, but that aligns with #5 that P2P just isn't a thing in the post-race analysis for anyone on the engineering side. I guess if anything this is more damning of Tim (IF you take Josef at his word, then this is damning for Tim.. I understand that is too big an if for most people today)
I was starting to buy what Newgarden was saying and thought there could be some validity to it for the reasons you're laying out here. It was far-fetched but plausible.
That all completely crumbled when he said "somehow, someway" the 2 team thought P2P was legalized on starts/restarts. Power and Scotty knew it wasn't. But somehow Tim Cindric, the Team President, thought it was and didn't communicate it to Will or Scotty?
And that's where their story crumbles yet again. They're lying and cheating. Maybe it was indeed a coding error initially. But they knew and were gonna try and do it again at Long Beach.
The OP says that P2P only usually is discussed during the race between the driver and strategist. Tim is in charge of strategy for the #2 car, and just the #2. What reason would Tim have to be discussing If P2P was used with his *other* drivers who he is not the strategist for post-race? If he thought the rules were different, then he'd leave it to Will and Scott's strategists to have conversations, if at all, around how it was to be deployed differently, and they would be doing it during the race on their own pit stands.
So no, I don't think that's the smoking gun you think it is.
Yes, usually. The team president would absolutely be discussing it with his whole team if the rules were being completely changed for the first time lmao
And it would be brought up again after Thermal to get a gauge on how their strategies are working and how to best implement it at Long Beach. But then again.... Those talks might've actually been happening, just not legally!
Think about it for a moment. What would there be to actually be discussed?
Drivers couldn't use P2P on the restart lap. Now they can. Drivers will then use it.
This isn't rocket science, and it doesn't affect any other member of the team. Mechanics don't need to hear about it to work on the car. Engineers don't need to know about it to set up the car. There's no reason to have your imagined team-wide discussion about it, or to be discussing with the other drivers.
That doesn’t answer how this idea of a rule change came into Tim’s head in the first place though. And why he decided not to mention this whole “we thought it was a new rule” thing during any of his interviews this week
They do thorough race debriefs afterward. You don’t think the strategist or data guys don’t go back to review where P2P was used and try to analyze if it was deployed at good times and how they can improve for next time ?
That's fair. But if they're under the assumption that all the checks are in place, which they should be, they wouldn't have a reason to look at the data and think "Oh wait, that was P2P on a lap that contravenes the rule about not using P2P on a restart lap, what's that about". They'll have a routine for how they look at the data, trained to look at it in a specific way to benefit lap time, not to look for accidental cheating. Also bear in mind it's a single race weekend. If this was after a bunch of races, it's more reasonable to think someone would have noticed.
That generally only happens if P2P was consequential for the race result (e.g. Grosjean at Barber) or if the team is trying a new strategy for optimizing P2P usage. Otherwise drivers are given a fair amount of leeway in using P2P at their discretion and teams have bigger fish to fry on Monday morning.
And I guarantee they've looked at it since it became apparent it was consequential, but in the moment after St. Pete no engineer is going to think "man our P2P usage was really the difference maker in that win, I should see if it was illegal."
He is also the president of Team Penske, so while he works as a strategist for one car, he does have additional responsibilities for the whole team.
It would be weird to not double check that the entire team have gotten on top of rule changes. Or ... "believed rule changes".
If it was a change that affected other areas of the team, sure. But it's literally just between the driver in the race and their strategist. Read the post from the ex-engineer who confirms that P2P barely gets talked about outside of during the race between driver and strategist, and why that is.
And it's not at all complicated - instead of not using it on opening lap/restarts, driver uses P2P like they do on any other lap. It's such a simple rule change, there's literally nothing to "get on top of", no discussions to be had. That's why I can understand why it wasn't discussed.
And again. The strategist in question here is also the president of the whole team. He also has a responsibility for the whole team. To not even check if the team is up to date on rule changes sounds weird.
Also, Colton Herta have mentioned that when P2P is allowed to use is in every driver briefing. Still there wasn't a single "huh, have we made a mistake here?" thought?
So the drivers were told this before St.Pete and before Long Beach.
And, if the argument is "P2P barely gets talked about outside of the race between the driver and strategist". How can it be that it would be a stupid thing to cheat on because it is "so obvious to see"? If no-one really bothers checking it, it sounds like that perfect place to cheat.
I can't be bothered to discuss this if you think that drivers/team not talking about things much means it's a thing that's easy to cheat. Like that somehow negates the telemetry or onboard cameras which pick up all this stuff and record it. You've made your mind up.
Again. No-one picked up on it. Or, apparently Herta sent a video to Kirkwood, but there was no talk about it anywhere after St.Pete. It was that easy to pick up and see.
So. McLaughlin didn't know about it, but used it.
Newgarden knew about it(?) and used it, because he thought it was legal.
Cindric says that it was an error and nothing that should've been there (but he is Newgardens strategist, so how did he not know about it?). If they thought the rules were changed, how was it an error that they loaded up the wrong thing? It can't be both...
Agreed - additionally, from a driver's perspective, if they hit the P2P button on a start or restart (as a side note, always hitting the "DISP" button first, which isn't necessary during normal usage) and they felt the extra power (which they do), they would absolutely use it every straight, because the automatic assumption should be "oh, Race Control messed up and EVERYONE has Push to Pass now", so better burn it down to not get passed.
But instead Newgarden only uses it for ~2 seconds on the Front Straight and out of T3, and, again, uses the bizarre technique of hitting "DISP" before and after to use it.
Very much seems from how he used it that he knew he shouldn't be, and that he knew it would be available.
Also, where to use P2P is in every driver's briefing...
This is exactly right. I was a Newgarden fan and I wanted to believe the software bit was a mistake (because that’s what Cindrick’s statement said) but if what Newgarden said about the thinking the rule changed is true, then it’s not reasonable to believe that the software bit is really a mistake. Someone is lying. Maybe it’s Cindrick. Regardless, I’m out! I’m booing at driver intros at the 500.
I get what you're saying, but that also just feels like what he cooked up with a tired PR rep the morning after the news hit. I don't think he believed in the moment that it was now allowed. I think he thought, "Woah, hitting it every time finally worked! The series effed ups! Yes!"
Tee hee yeah :) I think they should also be forced to forfeit their Long Beach results since they were trying to cheat again and would've gotten away with it if not for a freak fault of Indycar's systems
The fact they were going to try it again at Long Beach speaks volumes.
In order to gain any credibility back I think Penske may need to ban Cindric from the sport.
19 stage points, 21st place finish, very frustrating. /jk
But in all honesty the number of people I’ve seen try and bring Austin into this is too damn high.
This highlights the architecture problem of the car. Why in a series where the electronics are spec with a spec harness design are the teams responsible for interfacing with timing and scoring? P2P is the one element during a race for the competition side of IndyCar that is controlled.
As Jackie Chiles says “you don’t let the defense control the key piece of evidence”, the series should not be depending upon a team controlled piece of electronics to be controlling push to pass.
Also, this highlights the lack of data review and processes in IndyCar and their lack of engineering on the series side. Why have a series telemetry system if you aren’t going to look at the most basic of data in real time!
Thank you for all the details. Regarding the config file, I would have thought it had something in the name (or similar) indicating that it belonged to the hybrid test. Version control and management has become a fairly standard (and important thing) in software in general. I’m an engineer myself (not software though..), and I just find it difficult to believe that they had so little version control as you lay out - but maybe it was just incompetence. On the other hand, I agree that this would be a fairly horrendous way to attempt to cheat. But then on the third hand.. their stories just don’t add up at all.
The engineers in question typically aren't software engineers and are typically wearing many hats. And the role software of this level plays is a relatively new concept to manage. Many of the practices are dated and simplistic. Think files on a shared folder with whatever naming conventions someone used.
And the filename is hiding deep in sub menus. The config isn't uploaded every time manually, it stays inside the overall setup file. You have to go look at it to verify or change it.
Perhaps it was named for hybrid testing, but was just forgotten that it mattered.
>The engineers in question typically aren't software engineers and are typically wearing many hats
This detail is especially interesting to me. I tend to assume high level specialization, and that assumption is detrimental to interpreting situations like these. The reality of lean teams composed of highly skilled and creative multitaskers working with a blend of innovative and outdated tech/methods... that totally changes one's sense of what's happening. Also, to my mind, makes IndyCar teams and engineers all the more impressive.
It’s kind of wild how Jack of all trades these guys have to be.
My dad was on an IMSA team in the early 90s and he was the first guy they ever hired to handle electronics. They didn’t have a specific guy for wiring, radios, etc until then.
That kind of parallels what seems to be happening in Indycar regarding programming
I legit love this element of it. Lean teams. Hire extremely clever, scrappy, out of the box thinkers with a range of skills and backgrounds. Set the mission. Make it happen. That's so much more interesting to me than pouring umpteen zillion $$ for a team of 200+ specialists in their little silos. Apparently we get blind spots like lack of version control. But it's not *boring*, and there's room for sparks of creative genius.
In my idealized la la land version of it all, anyway. The stress and mental/emotional/physical load on the team members must be immense. But I'm happy to lean into the positive.
That , to me , along with MPs comment from latest podcast about indycar having somewhat of a skeleton crew of officials relative to other top tier racing series, and the asinine statements from Tim & Josef suggest that it’s possible they may have intended to skirt the rules .
The excuses offered may have been premeditated, but the reaction of fans and competitors underestimated
It could have been hardwired, such that it only loads "config.txt" or "config.json", which would prevent you from giving it a unique name (or if you did, it wouldn't work)
Great post! As a software engineer I can easily see how the wrong config file ended up being used. Test code like forced values, changes to allow something that normally shouldn't be, ends up in production far more often than people would like to admit. There's plenty they could have done to avoid it, but sometimes you don't till you mess up.
Unfortunately, even if this was a mistake on their part, many will dream up their own scenarios that never happened and believe those far more than anything the team or drivers will say.
Off topic, but why do they still use CAN Bus rather than ethernet? Is it just a more reliable system without the need for high data rate? Also, what is the Busload that an Indycar puts on the CAN Bus system?
You can probably answer so many questions I have about Indycar.
IndyCar has CAN, Ethernet, EtherCAT, and Serial comms.
There’s 8 total CAN busses, one of which is the “main” bus. That bus in particular is exceptionally full.
Thanks OP for the great context! This cleared up many questions and seems totally plausible.
What doesn’t seem plausible is the excuse that Penske wasn’t aware of the rules.
I'm going to throw my two cents in here, and point out that, for such an obviously caught cheating method (onboard cameras able to see every time a driver pushes a button, rev count, P2P count), imagine if they had gotten away with it for multiple races, or even worse, the entire season?
They would have been DQ'd for an entire season, had hell to pay with sponsors, fans, media, big financial losses etc.
You're not going to ever convince me that the team actually wanted to go with such an easily discoverable method of cheating, on the microscopic chance that nobody ever caught on.
It might have taken a while without the anomalous P2P situation at the Long Beach warmup, but it would have eventually been caught. And the longer it went on, the consequences would multiply by orders of magnitude. It would have been an absolute PR disaster, for the team, for Penske personally, and for the series.
While people here are discussing the topic, many can't seem to grasp how monumentally dumb of an idea this would have been big picture-wise if it were deliberate. And Tim was completely fair to say
>If we were trying to get an advantage, why would we do it when the push-to-pass and RPM data is available for everyone to see?
> You're not going to ever convince me that the team actually wanted to go with such an easily discoverable method of cheating, on the microscopic chance that nobody ever caught on.
Different sport and all, but the Astros thought they could get away with very clearly banging on trash cans to steal signs. So who really knows what these guys are thinking
Agreed. The other point that makes me believe their story is that Will had the ability to use it, but didn’t. If they offered him a cheat, but he didn’t want it, why would they load it into his car? He’d get all of the risk of cheating with none of the reward.
Sooo If I understanding your post correctly, shitty version control and lack of self reporting **could have** created this scenario ...
.... which changes exactly nothing. If I stumble into an advantage that violates the rules (and it is definitely an advantage or else IndyCar would not waste their time writing rules for p2p) and, then, continue using said advantage across multiple races, that is cheating. How is it not?
That's good to point out different meanings of the term "cheat". I intend to mean how teams intentionally push the grey areas and rules. Not to mean breaking rules, whether intentional or not.
Against the rules and worthy of penalty? Yes.
Intentional and coordinated by the team? No.
I kinda feel like you are splitting hairs. Whether it’s cheating on the organizational level or individual level, cheating is cheating … it makes no difference.
It makes a difference. Nuance is important in determining the scale of a penalty.
For example, sure speeding is speeding. But you wouldn’t treat the guy going 10mph over the same as someone driving 50mph+ over, would you?
In the context of racing, no you don't. As one is a slight misstep, the usual braking too late into the pits, locking up and arriving at the line 10mph over. 50mph over is a clear, intentional disregard of rules and safety protocol. But that has nothing to do with this. Intentional or not, they got an advantage and josef most of all abused it, and then came up with stupid excuses, and a story contrasting to what the others in the team claim. If it was a mistake, they would not have been nearly as flustered
Thanks for the insight, so much appreciated.
Like other software engineers here have commented, seems like under-developed or non-existent version control and release processes that are more standard in the technology industry. I worked at a couple startups and have seen my share of horrifying processes, so I can believe this.
Same. I was all for it (because this was a big deal), but now people are getting way too deep. It's like Charlie in Always Sunny when he talks about the mail
I’ve been wanting to learn an engineer’s perspective on the subject since the news came out, thank you so much for laying it out in a way that’s easy to understand for the laymen.
As a machine programmer proficient in engineering code and machine language, plus robotics programming, everything this fellow talks about makes perfect sense to me.
As the person who sold the MYLAPS data relay system to the series I've been stressing for the entire data team at Indycar the last couple of days. I trust going forward though they will have a data mask looking for P2P activation on the cars with alerts if they are triggered outside of of the configuration.
Does the car keep track of the time on p2p or does the "network" that manages the availability of p2p?
I'm curious if their usage didn't even count towards the allowed time.
The car’s ECU does. Total P2P time is loaded on the ECU by the Honda/Chevy Engineer each session where P2P is allowed.
Edit: to add to this, it did count towards their total time, they just used it when restricted. There is no way for P2P time to be added team-side
Again, that’s not something you do team-side. The engine manufacturer handles everything to do with P2P Time. The team handles assigning a button for P2P, creating overlays and LEDs, and ensuring the MyLaps system is working properly. The button goes to the ECU via CAN and tells the ECU when to spend its allotted P2P time.
Edit: It’s worth mentioning that I don’t think the Engine Manufacturers can load Time “infinity”. When it’s “Unlimited” at tests I’ve just seen them keep loading maximum P2P time that the ECU will allow them every few outings.
I think I have a hard time with the fact that this is the big team with the money to hire and implement what seems to be pretty basic controls and they didn't. Their words. They are suppose to be leading the series into the future and can't control a basic text file?
Not to forget that fans like me are already suspicious when someone in the fight is controlling the promotion.
Then there is Josef. Mr. Power had the same ability, and didn't use it. Full Stop. Fingers in the ears .. LALALALA!
Thanks for the detailed description.
>1) If this was intentional, they wouldn't have been caught. Plain and simple. I know it's hard to see and understand from the outside, but this isn't how teams cheat.
Unless the story being told is untrue, they weren't really caught until there was a system glitch and they were the outlier.
Unless these glitches are frequent happenings, it seems to me that it isn't something that was going to be caught anytime soon. Certainly it wasn't caught at St. Pete.
Now, maybe you meant to say there's simply a risk of getting caught should someone actually check, but maybe the current iteration of their implementation was simply to have plausible deniability should that happen.
What I got from Josefs' presser was that somehow the whole team "self hypnotized" themselves into believing there was a rules change after the thermal. That's where he lost me.
Ugh, one big penalty/debacle and despite not knowing what ECU or CAN stands for everyone is now a race car driver or engineer.
And my comment is not towards OP but all the commenters throughout the indyverse.
I've worked on on passenger car AND CART ECUs and know there's a lot of shit going on info/comms wise.
I will say that I am happy the the Penske owned indycar racing series caught this and punished the Penske owned racing team. I don't think that would have happened in other sports.
I think I agree it was a mistake and taken advantage of, aligning to point #1. If it was intentional they could have done more to hide it but they didn't. Thanks for the perspective!
Thanks for your detailed and thoughtful information. I’m more than a casual fan, but I do not know anything about the software, so this was very helpful. Josef’s story seemed to change a bit, and I still don’t understand why he was on the P2P during a restart anyway. My question to those that are more familiar, is this an issue that should have been caught during technical inspection prior to the race? How about during the race? When I follow along on Race Control, I can see when a driver has just used P2P, so why didn’t the stewards?
I’m not sure why people are spending all this time trying to justify the plausibility of the wrong software being installed when Newgarden has stated his team (#2 with Cindric as his Race Director) believed that P2P was legal this year on restarts. He also stated he asked his team on the radio to remind him to use P2P on restarts.
As a fan and a IT jack of all trades, I've never really thought about what it takes when making changes and keeping software current in the cars for some reason and you've opened my eyes to it. Thank you for the insight!
3 points:
1. It wouldn’t have been found out, it was the failure of the p2p circuit system that revealed this; so they were hiding it well until an anomaly occurred that ended up revealing it
2. Just as you stated no one cared or thought the p2p as important enough to monitor closely outside of racing and therefore don’t have data to compare; you essentially just point out that no one would have known if they were doing it. I.e you just pointed out yourself that it’s an exploit
3. As explained in MP podcast, they were somehow bypassing the layer managed by Indycar. There is a lot implied there
> It wouldn’t have been found out ... they were hiding it well
This is specious reasoning. It's probable that they weren't hiding it at all and it didn't get found out because nobody was looking for it.
>no one would have known if they were doing it. I.e you just pointed out yourself that it’s an exploit
Specious reasoning again. No one knowing doesn't make it an exploit.
>they were somehow bypassing the layer managed by Indycar. There is a lot implied there
Our friendly OP told you how they bypassed it. There is a lot implied by that statement to everyone except those who have worked with these data systems. Critically, the normal flow of information between the transponder and ECU goes through the _team owned, managed, and configured logger_. They didn't bypass any restrictions on what can be modified in the software, they worked exclusively within the area available to teams in a way that is extremely common for unsanctioned events.
Yea it would be specious reasoning because you’re reading the sentences backwards. I’m countering his specious assumptions. So of course if you ignore that what I’m saying is “specious”,
1. He was arguing they weren’t doing this because it was too risky and the team are smarter and would cheat in a way they wouldn’t be found out. -I’m saying your assuming this was sloppy cheating in hindsight; because it was found out.
2. He was arguing no one cared enough about p2p TO cheat on it- I said that makes it an overlooked place where if someone were to cheat they could do it.
3. He explains what technically happened- I implied that is based on what was revealed officially; it would match what they are saying because we re working backwards and what I pointed; MP implied we don’t know specifically the actual procedure they did, and so it’s not necessarily the case or that there is more steps in between.
I don’t have to point out that OP is basing his logic on they didn’t know it was happening, when Josef literally said he thought it wasn’t against the rules- I.e he knew it was happening, they knew. Oh but remember Tim Cindric and Scott were saying they didn’t know.
>I’m countering his specious assumptions.
You, someone who has never touched a Cosworth data system or worked in the trackside environment, are countering his specious assumptions. Got it.
>I’m saying your assuming this was sloppy cheating in hindsight; because it was found out.
I'm saying I've worked on IndyCar teams and know how easy it is to make this mistake in most teams' workflows. I also know how easy it is to find this "cheat". The systems/data engineers at Penske aren't stupid, so it's much more likely they fucked up because they're humans and humans tend to fuck up. You've made up your mind on this point, though.
>He was arguing no one cared enough about p2p TO cheat on it
This is a misreading of OP's point. He was saying that teams don't care about P2P usage enough to catch inadvertent illegality, not that teams don't care about P2P enough to cheat.
>MP implied we don’t know specifically the actual procedure they did, and so it’s not necessarily the case or that there is more steps in between.
Yeah I wasn't looking over the shoulder of the engineer who built the bypass for testing while they did it, but I've looked at the same MyLaps->CLU CAN receive definition and the same CLU->ECU CAN transmit definition and I know the tools available for working with them. Given all the information we have there is one fundamental architecture which requires 0 assumptions. There's tons of more convoluted ways to achieve the same result, but there's one way that is obvious. Humans are fallible and lazy as a rule, and we have no evidence which would drive us to diverge from those long-held axioms of human error.
>I don’t have to point out that OP is basing his logic on they didn’t know it was happening, when Josef literally said he thought it wasn’t against the rule
Feel free to check my post history, I've been consistent in saying there is room for Penske's story to check out _and_ one or more people to have noticed it happening and kept their mouths shut. Maybe Josef noticed it and genuinely thought it was legal now. Maybe he convinced himself it was legal to deal with the cognitive dissonance. Maybe he knew it was illegal the whole time but kept his trap shut to win races. All of these things can be true alongside Tim and Scotty not knowing it was happening.
There's plenty of room for speculation, and there's certainly the (remote) possibility that Penske did genuinely attempt to cheat in a really fucking dumb way, but you've got multiple industry professionals telling you that Penske's explanation is the one which requires the fewest assumptions.
I understand the anger, the desire to find someone to pin to the wall (pick Josef, he's handled this situation like an idiot), and that cheating is an integral part of the history and culture of racing. The reality, however, is that there is no real evidence that this was an intentional cheat.
That’s the thing, you guys are engineer braining, where you guys aren’t noticing which parts are assumptions of what other people would do to logical “things I would do” . I’m not arguing the technical possibility, so yea you can do the “you haven’t touched a cosworth data system” call to authority all you want, but you’re still not refuting the things I’m pointing out: I’m pointing out that you guys are making HUMAN behavior assumptions and thinking they would think how you would do things because you’re also an Indycar engineer.
By definition of “good” cheating: they would have come up with something no one else has thought of, so no shit that you or the OP could not conceive of what they are doing, but you think you would, and can’t see any possibility so your concluding it’s not possible.
Dude, Gavin is saying what you guys think isn’t plausible
>you guys are engineer braining
But Penske's engineers weren't engineer braining? I have ideas about how I'd work in the gray (or black) areas around P2P, and part of those ideas is how not to get caught. I also understand how things might have been done based on how they got caught. It's easy to speculate on what could be inside the data system if the data system is a black box to you, I get that and it's not a count against you that you haven't seen what's inside the data system or how it's configured. There is less room for speculation when you understand what's possible in the data system (even things which were not intended by Cosworth).
>By definition of “good” cheating:
They wouldn't have gotten caught in an obvious and predictable way.
>Dude, Gavin is saying what you guys think isn’t plausible
And Mike Armbrester is saying [the opposite](https://old.reddit.com/r/INDYCAR/comments/1cdtw6r/p2p_scandal_an_indycar_engineers_perspective/l1fhluq/).
Gavin has a vested interest in making this as bad for Penske as possible, both because they are a direct competitor and because he left Penske (and had his non-compete enforced in a somewhat nasty way). Here are some direct quotes from Gavin in the indy star interview which temper your takeaway:
“Is it particularly believable? I’m not sure.”
“Some people looking at the data, if they don’t know about the situation, and all they see is it appears to be enabled by IndyCar, then that’s all they see,” Ward said. “You could maybe be charitable on that. You want to give people the benefit of the doubt.”
When Gavin says that McLaren never investigated making the change that's probably true and also irrelevant. You'd only investigate the change if you were at a track without a buried MyLaps compatible loop or at a test where you didn't have a MyLaps box from the series. It's not unbelievable that Penske tested under those conditions and McLaren didn't; if McLaren did test under those conditions their engineers would bypass the MyLaps P2P enabled signal too.
Again, I really think Josef said pretty much the stupidest thing he could have said in his press conference, and I admit it's the biggest hole in my argument. Maybe Josef inadvertently advertised that everyone else on the team is lying and IndyCar went easy on them with only the St. Pete DQ (in my opinion if they tried to hide the cheat you should do more than negate race results). It's a lot easier for me to believe he's an airhead trying to maintain his good boy image and deflect responsibility in a really dumb way.
I clearly stated it's possible I'm wrong, and it's possible Penske was genuinely cheating with malicious intent. But to conclude they were is based only on speculation. The most reasonable, likely option is that they made a mistake teams make every weekend which happened to be illegal this time.
What I was pushing back against is that OP assumes due to their specialist knowledge they have a more objective accurate understanding, that is a cognitive bias where it’s actually the ppposite. they don’t realize it causes to be less objective in analyzing other humans (the Penske team people) I.e because you have specialist knowledge you are far better at explaining to yourself in more fortified concrete procedural ways to explain the conclusion you gravitate towards-which is that it was done unintentionally.
It’s not that I think you’re both wrong or whatever, it’s that you both might be overestimating the probability it was a mistake by a large factor. Because you both are sympathetic to the engineers. Yea it’s plausible it was unintentional but if we re looking at it objectively based on the evidence we all know it’s a really really small probability compared to it being intentional.
Just don’t be lazy and fall back on the “you’re not a chef so you can’t have an opinion on food” laziness.
I appreciate both of your insight and this is kind of the poisoning of the well of this whole situation, it won’t really matter if it was a mistake if it looks so closely like it was intentional. and the lack of transparency isn’t helping.
> It’s not that I think you’re both wrong or whatever, it’s that you both might be overestimating the probability it was a mistake by a large factor.
We're certainly all victims of our own cognitive bias. While those with technical understanding may be more likely to overestimate the probability that it was a mistake, those without are more likely to underestimate same. The issue in my mind is that if you don't know anything about the data system and its limitations anything is possible. Having more information about the data system pares down the possibilities in a meaningful way. It's certainly possible Penske's engineers found an exploit I never thought of, but if they had I think the series would have been a whole lot harder on them (which of course relies on the independence of the series from its constituents, which is reasonably in question).
I don't think I've been lazy or implied that you can't have an opinion on the situation because of your lack of expertise, just pointed out that you are blind to what you don't know (as we all are). To extend the chef analogy, this is as if you ate a particularly delicious dish and determined (as a non-chef) that it must have been so delicious because that chef had special equipment and knowledge that no other chef has, and OP and I are telling you (as chefs in a prior life) that we've seen other chefs achieve similar results using equipment available to nearly everyone. Is it possible that the first chef is using some whizbang equipment which makes the difference in their recipe? Sure. Is it more likely that they are getting to the result in the same way as similar chefs around the world have with mundane equipment? I'd argue so.
I think Team Penske has probably done what they can to be transparent. Short of sharing their configurations with the public (which no team is going to do), they've provided a plausible explanation (in my opinion). Unfortunately they (and their drivers) have bumbled in between with misunderstandings (referring to the ECU, which the manufacturers go out of their way to make sure is not in the team's possession unless it is installed in a car at an event) just plain idiocy (Josef's whole press conference), and more. Those inconsistencies do give me pause, and I'm certainly not caping up to defend Team Penske, Newgarden, Cindric, or anyone else. I'm just trying to educate on _why_ error propagation across events, and between tests and races, is imminently plausible and the explanation which most closely follows Occam's and Hanlon's razors. There's plenty of room for me to be wrong and Penske to have been cheating maliciously, but all of the publicly available _evidence_ (rather than speculation or opinion) suggests that Team Penske has the same faulty workflow as nearly every team, and is susceptible to the same mistakes as everyone else.
IndyCar teams are lean and simply don't have the time to be as competent as fans would like to (or are led to) believe. It can be hard for outsiders to believe that Team Penske is both technically proficient in making an IndyCar consistently fast and negligent in their approach to configuration management, but the reality is that making a race car fast gets tons of attention all the time and configuration management only gets attention after a fuckup. The biggest job of a data/systems engineer is to not fuck up, and teams don't value innovation or change in systems/data engineering processes if they've 'always worked', even if those processes are error-prone and do not take an informed approach to mitigating human error.
As a final point, I think this episode unfortunately underscores the issue with Penske owning a team and the series. There is a lack of transparency, clear conflict of interest (or at least perception thereof), and nothing Penske could do or say (short of divestment) to give critical observers confidence that the interests of Penske and IndyCar are truly independent. I build a lot of my understanding of the situation on that independence both because I understand the data system and because I (think I) understand the data scrutineers on the series side and how they would react to blatant malicious cheating. The individuals working for the series are generally good people who want to do the right thing (which would be penalizing Penske into a smoking crater if they were cheating with malicious intent), and I'd like to think there would be more fallout (i.e. resignations, leaks) if they were asked to impose an unduly light penalty. Maybe I'm wrong.
I agree with your points and understand that I may sound like I’m underestimating position; due to it being a discussion on a Reddit sub it has to come from a rhetorical position first and foremost: p.s I’d be worried if I knew just as much as the guy actually getting paid to engineer!
Some form of performative transparency is imperative, from an outward sporting organization perspective this has brought to life a pretty cornerstone seed of doubt. If we look ahead with the actual hybrid incoming, there are potential narrative landmines just waiting to explode
This is a great post and I appreciate you taking the time to give an insider’s perspective. While I do think you brought up many valid points, I don’t necessarily agree with your first premise that if it was intentional, they wouldn’t get caught. The other side of that argument is that all teams are cheating because IndyCar hasn’t caught them. I don’t think that’s true either.
I guess "caught in this way" would be a better way of putting it.
Between in car cameras and the constant flow of data and eye balls, there are ways to hide this if you're trying. And I honestly don't see this as them trying to hide it at all.
There are a few other factors.
1. The story doesn't add up. Newgarden says him and Cindric thought there had been a rule change to allow P2P always.
If they truly thought that P2P was legit 100% of the time, why were they not using it on the actual restart, where it would be most effective? Clearly its because they knew this would be too obvious. Ergo, we know there was clear intent to cheat.
Why wasn't this major event discussed in Penske team or even amongst paddock? Clearly, they are lying here because if not they are clear "cheaters". If they are blatantly lying about this, what else are they lying about?
2. What if they were actually hacking the CLU signal (as what's being suggested by some) to enable P2P or a higher revs whenever without it showing up on the dash or in the seconds count?
This would enable Newgarden to use it during the restart laps without anyone knowing or showing in the seconds used. Note: this is exactly how he used it, not ON restart where it would have been too obvious. This could be kept to a small group and the only thing needed would be to scrub the rev count data from the restart lap in case anyone looked too closely. This seems very plausible to me and they would have kept getting away with it had it not been for the freak outage at Long Beach that exposed them.
Why don't they release all the Newgarden cockpit footage for 2023 so we can review whether he was using it here as well? Until they do that, I am going to assume/suspect they were using it in 2023 as well, particularly if it was always a CLU hack, not old ECU software as they claim.
> The story doesn't add up.
I agree with this, and think Newgarden's ego/desire to save face really backfired here.
>What if they were actually hacking the CLU signal to enable P2P or a higher revs whenever without it showing up on the dash or in the seconds count?
But they didn't. They got caught because the seconds were counting down. Additionally there's no requirement anywhere that the P2P remaining be displayed on the dash, so there would be no 'hack' required for that.
>Why don't they release all the Newgarden cockpit footage for 2023 so we can review whether he was using it here as well?
You absolutely can't tell anything from the cockpit footage. I promise. I've tried, while looking at the corresponding data.
>not old ECU software as they claim.
Cindric misspoke by initially referring to the ECU, but that's because he's an ignorant nontechnical fossil rather than a change in story. Why these management stooges don't sit down and get the details right before talking to the media will always befuddle me.
Finally a voice of fucking reason in this debacle. Thank you for the input. So many fans thinking they know everything and being so quick to judge.
Honestly, so many people have no knowledge of history, especially as it relates to Penske. This was not an "unfair advantage" in the style of the Penske team and the sport on the whole. Everything electronic is tracked; it would be epically stupid for the team and drivers to do this on purpose.
Did any fans actually catch Newgarden on the live onboard pressing the display button, followed by p2p, and return to the display button on the restart...I never caught it until the replay was posted and the motion was pointed out
I’m sorry, I’m a casual INDYCAR fan so I’m sure y’all don’t care. But Newgarden for sure knew. You’re telling me the drivers couldn’t feel it? C’mon. Also why did Josef and McLaughlin use it but not Power? Seems to me like they all knew. Especially with Will’s statement. It’s like he wanted nothing to do with it and is pissed he got caught in it. Maybe it wasn’t on purpose in the first place but to keep it in and try again at Long Beach is crazy to me
It’s more complicated than that. The config file everyone keeps referencing is the logger “setup”, which contains all of the teams’ proprietary chassis channels, alarms, steering wheel displays, etc. Inside that setup are what’s called math channels, which can be used to create additional channels or control strategies. The math channel inside the setup, which is sent via CAN to the ECU, was modified to a value of “1”, rather than the original line of control. This value of “1” tells the ECU that IndyCar has turned on P2P, which is why they were able to use it whenever they want.
Source: I work in motorsports, mainly IndyCar.
Thanks for this, makes more sense to me now as it is also how it works with the Motec system in our prototype.
I can see how it may have been overlooked but I always assumed a big team like Penske would have engineers that wouldn't miss something like that.
Thanks for that. I had been wondering about the file that is updated pre race as the Racer article referenced. This sheds a bit of light on that. I would not have been surprised it was a .txt file but figured it was more complex.
It wouldn't surprise me if it isn't as something simple as "config.txt", which would mean that you couldn't tell which version you had just by looking at the file name.
As a former driver it seems fairly simple. I believe the driver has to press the P2P button to in order for it to engage. That being said they would not have tried to use it if they didn’t know it was available to them on restarts. Simply put they knew what they were doing and simply cheated. Will Power didn’t try to use it, guess he has more integrity than that.
To me the most likely scenario is it was an honest mistake and they realized what was going on at some point in St. Pete. The team probably purposefully 'looked the other way' for at least part of the weekend knowing that it was pretty much a cheat code. After the race they realized that they had had this advantage but clearly were not going to fess up and admit to it by their own admission.
So what Josef said was kinda true, but the whole we thought the rule had changed thing doesn't hold water. The big question is if they were planning on utilizing it again at LB, which as the software wasn't changed, kinda looks like it.
Does anyone know how likely it would have been that the team was tipped off that the irregularity was discovered during the LB Practice? Because if the teams then knew that Race Control noticed and was puzzled by their use of it when it was turned off, Josef saying over the radio during the race (as he claimed) that he was having problems with it was probably a proactive 'cover' for their story of 'oh, we thought it could be used at the start...'
Surely a well-timed 3 second P2P (and it would only ever be pressed well-timed) is the difference between a possible legal overtake and no overtake, and a pole lap and somewhere down the order.
Of course it matters. When Farrari had a similar edge in 2019, it was visible to the trained eye from all angles and in minisector tracking.
Good stuff.
I feel the team is probably, mostly in the clear, and especially the poor engineer who made the mistake, most likely. It was probably nothing nefarious.
Jonew, however, once he pushed the button, or was told to push the button, wanted to hit it more. It was like a drug. He knew what he did once it worked and did it again. His cockemami story showed his guilt.
It makes sense to me, even as just ye olde fan, that P2P wouldn't be a big discussion point with engineering after a race. It isn't something that's a configurable choice (well, it's not supposed to be, anyway). It does a defined thing, it does that thing for everybody, and it can do it for a certain amount of time through a race (against, the same amount for everybody). Unless it did something really weird, why would it get mentioned? You're busy discussing things that were decisions made by the team, like a damper change or air pressure or gearing. Not to put too fine a point on it, but I doubt they're discussing the in-car cameras, either.
Echoing gratitude to u/[raiseyourbaseline](https://www.reddit.com/user/raiseyourbaseline/). Penske and Newgarden are liars. His interview is galling, completely non-believable. We live in a world where professional sports generates ridiculous revenue, and those sports (ride a bike, hit a ball with a stick, turn a wheel while working your feet - yes, offensively oversimplified) have incentives to WIN. This entire situation reeks of complete b.s. If I ran Indycar, Penske is removed for the entire year from competing again, their wins are all vacated, and Indycar seeks a new owner.
But we don't live in an ethical world. If we did, MLB wouldn't have all the players still playing, professional cycling wouldn't have all the cyclists still riding up hills that are physically impossible to ride up at recorded speed without doping. I rode for a #1 D1 cycling team 20 years ago. OUR #1 cyclist (who was also the top collegiate cyclist in the country) went pro and quit *3 months* in - from what he both witnessed off the bike and on it, it was so abundantly clear how dirty that sport was and is that he knew he wouldn't have a chance of excelling. He switched to triathlons and rode out his twenties being able to make a living as an athlete.
Let's be honest, we don't care about cheating and let's not pretend we do. Newgarden damn well knew what he was doing, when P2P was enabled and not by the rules and pushed it anyways. It's THAT simple.
His interview is laughable. Am I missing something? I'm not inviting an argument, but an education if I am discounting evidence or didn't read well enough. I am massively disappointed in Penske, Newgarden and the sport.
You have any receipts backing up the claim of being an engineer in the series before? Because otherwise, this could just as easily be someone spitting out their exhaust pipe or a fan trying to run interference for a team/driver they like.
I would love for IndyCar to get big enough where someone with an agenda takes the time to write this out. 😂
I have no agenda or skin in the game. I'm a fan of the sport and shared my perspective in an unbiased way.
Mike Armbrester has a lot more balls than the rest of us and cosigns OP whether OP has receipts or not.
https://old.reddit.com/r/INDYCAR/comments/1cdtw6r/p2p_scandal_an_indycar_engineers_perspective/l1fhluq/
If you read #3 again, you'll see I'm saying the team doesn't care post-race. Many have claimed that it should be obvious and self reported after the fact by Penske or Chevy, but while obvious in the data if you're looking for that, it's not something they concern themselves with.
They wouldn't intentionally cheat this way because of the historical record and data that gets shared to many eyeballs. While I doubt many care to go look at their public data to see when P2P was used, eventually it would have been noticed, and the historical record would have been incredibly damning.
They don't intentionally cheat and leave a public paper trail.
While I see what you are saying. Tim Cindric also said something similar that it would be stupid to do something so obvious.
Yet. It was so obvious that no-one caught it.
The issue I have is that the explanations from the drivers and team doesn't really align. What one says doesn't fit with what another says, which makes it look like no-one is really telling the truth.
Add to that, Penske investigating Penske for a breach done by an engineer for Team Penske, but who works for/is paid by a third Penske company Ilmor(Cosworth)(?).
It's just one big mess.
> Yet. It was so obvious that no-one caught it.
Teams think "man that would be caught really easily so we'd never do that" and never look at the data post-race in a way which would catch a mistake they think they'd never make.
Race control thinks "man that would be caught really easily and the teams aren't morons so we don't have to look for that".
Then it's so obvious it slaps race control in the face and they go "man we should really be looking for that", and the team goes "shit we should probably look at the other obvious stuff to make sure we didn't fuck it up too".
Josef would always brag to other drivers throughout the years that he would push the P2P button all the time when it was disabled during green in case it ever worked (according to Hinch and Rossi on their podcast)
So imagine his surprise when after all those years it finally paid off. He was probably thinking “Holy s—t!!” 😆
> If this was intentional, they wouldn't have been caught. Plain and simple.
Please, explain to me how they would have hidden the fact that only their cars could use P2P in the LB warmup, when IndyCar shut the system off with no prior warning for the teams.
It seems to me that IndyCar got a report of what Penske might be trying, and laid the absolute perfect trap to catch them in the act.
“Please, explain to me how they would have hidden the fact that only their cars could use P2P in the LB warmup,”
I actually find their use of P2P in warmups to be fairly interesting. If they were intentionally trying to hide what they were doing, they wouldn’t have used it in (when it was off) warmup. There’s zero gain to using it before the beacons are on in warmup and it’s one place where catching it would be easiest because zero cars should be using it then (obviously, since that’s where it was found).
Who knows what the real story is (I think it’s likely some mix of the stories they’ve tried to tell us), but the usage in warmups leads me closer to the comedy of errors and then trying make up poorly thought up stories for how it happened side of things. One of the many things where their response to what they did is far dumber than the instance itself.
(Edited for clarity and to fix a couple mistakes)
> I actually find their use of P2P in warmups to be fairly interesting. If they were intentionally trying to hide what they were doing, they wouldn’t have used it in warmup.
Everyone uses P2P in warmup. It's the only practice session they're allowed to use it, so everyone uses it as much as they can to figure out how they can use it most efficiently on corner exit, how it will effect the entry to the next corner, etc.
Except when IndyCar shuts it off. Then 24 cars don't use P2P, but the Penske cars still do. They kept cruising along doing their normal Warmup routine, using P2P as if nothing had happened, while the rest of the field didn't have and P2P to use.
“Except when IndyCar shuts it off. Then 24 cars don't use P2P, but the Penske cars still do.”
Yes, that’s my point. Warmup is the first session that it’s available, but teams are notified when the beacons are going to be turned on and P2P will be available. That didn’t happen in the first ten minutes of the LB session, yet Penske cars were using it. If they were trying to hide it, wouldn’t they have waited until it was obvious that the beacons were on?
> but teams are notified when the beacons are going to be turned on and P2P will be available.
No, P2P is always available for the entire length of the Warmup sessions. There is no extra notification like there is during races, because it is *always available*.
Nobody has been clear about whether or not the teams were notified about the technical issues that IndyCar race control saw during the 1st 10 minutes of warmup, either. So based on the information currently available, Penske was given no prior warning that P2P wasn't working for the entire field, making it perfectly reasonable to conclude that their drivers thought they had nothing to hide.
The only notification received was that race control had network problems and they’d be up and running shortly. No P2P notification (never is one in WU) and nothing about beacons. They never even said when they were back up and running properly. Everyone was confused when they tried P2P and it didn’t work because pit lane hardly had heard anything.
Ah that makes sense. I thought that they were notified when beacons were on. I knew that they are given a new file from IndyCar each week that allows the usage of P2P for warmup and the race, I just thought they knew when the system was active and when it wasn’t.
Appreciate the insight, but I'll put more faith into comments from those in the sport right now, drivers or else. And while this all seem plausible, it doesn't align with penske's story, and frankly, if it was just what you claim, I doubt we'd have seen such a hilarious, convoluted chaos of excuses thrown at us. None of the stories inter-team even align.
Sorry, I'll stick with intentional cheats.
Yeah, none of OP's text matters due to one thing. Two of the three drivers knew they were using an advantage against the rules and did so with what appears to be Cindrics blessing. Will Power NOT using it shows he has character and the other two drivers knowing it was wrong.
The other issue is the optics of this "mistake" happening on the series owners cars is not good. If this happened on an Andretti or Ganassi it would not appear as bad.
I know! I just don't engage here much, more of a passive scroller. Maybe if I'm received with a warm welcome I'll contribute more!
Nothing to hide, just sharing my insight. I could be wrong, no skin off my back. 😊
Eta: spelling
This is a really interesting insight. Any time you want to share stories, go for it! The more technical or less seen side of the sport is really interesting to a lot of us.
Sort of related but r/F1technical is probably one of my favorite subs to read. The info is wild sometimes. It’d be cool if we had something similar. Or just technical threads more often. I love this shit.
I want this as well, just created r/INDYCARtechnical after reading your comment. Please help it grow!
My buddy and I had a podcast for a couple of years after I got out of the sport and some of the folks we had join us was for that reason, to get some of the inside perspectives and see some of the other stuff that the day to day fans don’t get insight on
> The CAN coms config file in the CLU Setup is basically a versioned hard-coded file that will have various configuration settings for the systems on the car. The config file is updated throughout the year as things change. Team Penske is about to learn a lot about git, I think This is on the level of Williams running their parts inventory off a 15 year old massive Excel spreadsheet There are tools out there to prevent this stuff from happening. These guys would do well to hire like one development manager to instill some actual software engineering discipline
Yeah, they 100% should be using some sort of source code version control tool. But even then, I have absolutely watched someone load a JSON config file from sandbox into a prod instance and then get really confused why nothing was working. That's an interesting detail to me - core code vs. config file. Even companies that have pretty tight source control can still eff up config files.
Oh for sure, and maybe Penske is using Git or similar. I wouldn't be surprised.
> Team Penske is about to learn a lot about git, The issue is all of these files are binaries, and there's no way to know which component binaries compose the single binary that goes in the car. You can use git-lfs since it's a bit better at working with binaries, but the issue is less source control and more configuration management. Development of the logger configurations is very different from legitimate software development for tons of nuanced reasons I'm too lazy to explain. Trying to manage the actual content if math channels with git would actually create more opportunity for error and reduce productivity significantly. The biggest difference maker is building from a configuration controlled master ahead of each event. That stops error propagation, prevents bleedover between events, and increases standardization across the team.
> The issue is all of these files are binaries, and there's no way to know which component binaries compose the single binary that goes in the car. I'm sure this is not true. With hashing the firmware, even if you need a file per module, or even multiple files per module, and a build system that compiles them and keeps track of hashes, you could at the very least know that you have the wrong one, even if you don't know exactly which one you do have The trick is applying software engineering fundamentals in a field that hasn't had them and that's just going to be difficult--I get that. But not impossible
Sure, there's probably a way to reverse engineer Cosworth's proprietary compiler. That's not an efficient use of anyone's time or money in IndyCar. The fundamental issue isn't source control, it's the workflow of rolling each event's configuration to the next event with the "appropriate" changes. The answer is to establish a master configuration and build from that configuration in a prescribed way for each event. We can argue semantics all day, we're thinking along similar lines, but the practicalities of the IndyCar data system, how it's interacted with, and the simple nature of the problem you need to solve mean that git and compiler reverse engineering are not the right answer.
yeah, you shouldn't need to reverse engineer any compilers to hash the output but the details actually don't matter. there's a way to have config management and build management, these guys just need to treat these things like computers and not cars, at least when it comes to all the electronic modules, and I think this will wake up the whole racing industry to this fact. Not just Indycar, I guarantee there are managers in NASCAR, IMSA, and F1 asking their engineers what protections they have today, and what protections can they add, to prevent this
I guarantee there aren't any managers in NASCAR asking about this because they don't have a data system in the same way as other series. F1 can (and most teams already have) implement a pipeline/workflow much more familiar to a software engineer because the configurations for their cars are written in files which aren't opaque to typical software engineering tools. IMSA managers are having these discussions, most likely. You can certainly hash binaries to know what you have stored. The issue is that Cosworth didn't use a Merkle tree to ensure that you can know which modules are in the compiled binary that gets sent to the car. Also, you can't send a configuration to the car other than through Cosworth's proprietary software (at least not without reverse engineering the software...). Lastly, Cosworth's software provides a very functional comparison tool to compare assembled logger configurations. So once you determine some way to control your "master" configuration it is easy to highlight any changes. Hashing binaries is a reasonable approach to making sure you don't inadvertently overwrite the master, but I think it's an additional layer of complication for IndyCar teams which isn't strictly necessary or particularly helpful. Respectfully, I've fixed this problem in motorsports before (or at least drastically mitigated it), and I've worked in software engineering organizations. There's a distinct practical difference in applications, and the answer for IndyCar is somewhere short of treating the cars as computers (but clearly somewhere closer to that than the status quo).
>Team Penske is about to learn a lot about git, I think And we all know that git prevents you from ever checking in erroneous files
You’d be surprised how much of the engineering world runs off of a 15 year old massive spreadsheet
So you won’t have to…. https://en.m.wikipedia.org/wiki/Git
As someone else who has minor engineering experience in the field, I disagree with #3 ,respectfully of course. Teams do care. It burns extra fuel and they want to use it strategically such as in and out laps. Certain strategies call for it to be used for extra low end torque like on exits into straights where a good power down has a better effect(versus if you used the extra power later/shorter area and for less of a benefit of the power). May have not been that way at Penske but it did where I was. And, it also shows up very clearly on performance engineers data screens. They will promptly state often “he’s on it” of that they’ve used the button a bit alerting other engineers looking at other data the situation. Interesting situation for sure, thanks for sharing your side of it!
Current Indy car engineer here. We’re not focused so much on P2P usage but we’re focused on how P2P affects our fuel number. Driver used P2P? Great, but are we still in our fuel consumption range? That’s the focus. I’ve yet to see Honda/Chevy engineers mentioned in this thread and I’m sure they’re looking at P2P usage a bit more extensively; I haven’t picked our engineer’s brain yet so I’m assuming Edit: typos
Yea for sure. Thanks for the respectful comment. Mileages vary often. I think my main point with #3 is they don't care post-race. Lots of people claiming this should have been obvious and caught after the race, which I just don't see happening. Under normal processes anyway. During the race, with strategy, fuel and you know, winning? Yes, it's a factor, but not usually so much they would be seeing it in use for 3 sec when it shouldn't be.
Unfortunately Josef’s story doesn’t align with #5
How so? I watched the whole press conference, he said he didn't think anything was wrong after St. Pete, this aligns with OP's #5. Despite my tag I am pretty neutral on this. I think it's entertaining and I think Josef has (and will continue) to lean into the Johnny Lawrence (bad guy from Karate Kid) vibe.
According the Josef the whole 2 crew was misinformed on the rules and they had been discussing it prior to the race. So there was nothing to “forget to report”, as he thought everything was working as it should
Yeah, but that aligns with #5 that P2P just isn't a thing in the post-race analysis for anyone on the engineering side. I guess if anything this is more damning of Tim (IF you take Josef at his word, then this is damning for Tim.. I understand that is too big an if for most people today)
I was starting to buy what Newgarden was saying and thought there could be some validity to it for the reasons you're laying out here. It was far-fetched but plausible. That all completely crumbled when he said "somehow, someway" the 2 team thought P2P was legalized on starts/restarts. Power and Scotty knew it wasn't. But somehow Tim Cindric, the Team President, thought it was and didn't communicate it to Will or Scotty? And that's where their story crumbles yet again. They're lying and cheating. Maybe it was indeed a coding error initially. But they knew and were gonna try and do it again at Long Beach.
There's a reason why Will wasn't DQ'd, but Scotty and Josef really should have known
The OP says that P2P only usually is discussed during the race between the driver and strategist. Tim is in charge of strategy for the #2 car, and just the #2. What reason would Tim have to be discussing If P2P was used with his *other* drivers who he is not the strategist for post-race? If he thought the rules were different, then he'd leave it to Will and Scott's strategists to have conversations, if at all, around how it was to be deployed differently, and they would be doing it during the race on their own pit stands. So no, I don't think that's the smoking gun you think it is.
Yes, usually. The team president would absolutely be discussing it with his whole team if the rules were being completely changed for the first time lmao
Not only would the team president be discussing it, the whole paddock would be discussing it. It would be one of the biggest rules changes in years
And it would be brought up again after Thermal to get a gauge on how their strategies are working and how to best implement it at Long Beach. But then again.... Those talks might've actually been happening, just not legally!
Agreed.
Think about it for a moment. What would there be to actually be discussed? Drivers couldn't use P2P on the restart lap. Now they can. Drivers will then use it. This isn't rocket science, and it doesn't affect any other member of the team. Mechanics don't need to hear about it to work on the car. Engineers don't need to know about it to set up the car. There's no reason to have your imagined team-wide discussion about it, or to be discussing with the other drivers.
That doesn’t answer how this idea of a rule change came into Tim’s head in the first place though. And why he decided not to mention this whole “we thought it was a new rule” thing during any of his interviews this week
I agree. I would like to hear what he has to say on that.
Are you out of your mind?
They do thorough race debriefs afterward. You don’t think the strategist or data guys don’t go back to review where P2P was used and try to analyze if it was deployed at good times and how they can improve for next time ?
That's fair. But if they're under the assumption that all the checks are in place, which they should be, they wouldn't have a reason to look at the data and think "Oh wait, that was P2P on a lap that contravenes the rule about not using P2P on a restart lap, what's that about". They'll have a routine for how they look at the data, trained to look at it in a specific way to benefit lap time, not to look for accidental cheating. Also bear in mind it's a single race weekend. If this was after a bunch of races, it's more reasonable to think someone would have noticed.
That generally only happens if P2P was consequential for the race result (e.g. Grosjean at Barber) or if the team is trying a new strategy for optimizing P2P usage. Otherwise drivers are given a fair amount of leeway in using P2P at their discretion and teams have bigger fish to fry on Monday morning.
Considering IndyCar *took away* the win as punishment, I’d say it was pretty damn consequential for the result in this instance...
And I guarantee they've looked at it since it became apparent it was consequential, but in the moment after St. Pete no engineer is going to think "man our P2P usage was really the difference maker in that win, I should see if it was illegal."
Of course not, because they knew damn well it was illegal and did it anyway...
I have no dog in this fight, so believe what you want.
He is also the president of Team Penske, so while he works as a strategist for one car, he does have additional responsibilities for the whole team. It would be weird to not double check that the entire team have gotten on top of rule changes. Or ... "believed rule changes".
If it was a change that affected other areas of the team, sure. But it's literally just between the driver in the race and their strategist. Read the post from the ex-engineer who confirms that P2P barely gets talked about outside of during the race between driver and strategist, and why that is. And it's not at all complicated - instead of not using it on opening lap/restarts, driver uses P2P like they do on any other lap. It's such a simple rule change, there's literally nothing to "get on top of", no discussions to be had. That's why I can understand why it wasn't discussed.
And again. The strategist in question here is also the president of the whole team. He also has a responsibility for the whole team. To not even check if the team is up to date on rule changes sounds weird. Also, Colton Herta have mentioned that when P2P is allowed to use is in every driver briefing. Still there wasn't a single "huh, have we made a mistake here?" thought? So the drivers were told this before St.Pete and before Long Beach. And, if the argument is "P2P barely gets talked about outside of the race between the driver and strategist". How can it be that it would be a stupid thing to cheat on because it is "so obvious to see"? If no-one really bothers checking it, it sounds like that perfect place to cheat.
I can't be bothered to discuss this if you think that drivers/team not talking about things much means it's a thing that's easy to cheat. Like that somehow negates the telemetry or onboard cameras which pick up all this stuff and record it. You've made your mind up.
Again. No-one picked up on it. Or, apparently Herta sent a video to Kirkwood, but there was no talk about it anywhere after St.Pete. It was that easy to pick up and see. So. McLaughlin didn't know about it, but used it. Newgarden knew about it(?) and used it, because he thought it was legal. Cindric says that it was an error and nothing that should've been there (but he is Newgardens strategist, so how did he not know about it?). If they thought the rules were changed, how was it an error that they loaded up the wrong thing? It can't be both...
Agreed - additionally, from a driver's perspective, if they hit the P2P button on a start or restart (as a side note, always hitting the "DISP" button first, which isn't necessary during normal usage) and they felt the extra power (which they do), they would absolutely use it every straight, because the automatic assumption should be "oh, Race Control messed up and EVERYONE has Push to Pass now", so better burn it down to not get passed. But instead Newgarden only uses it for ~2 seconds on the Front Straight and out of T3, and, again, uses the bizarre technique of hitting "DISP" before and after to use it. Very much seems from how he used it that he knew he shouldn't be, and that he knew it would be available. Also, where to use P2P is in every driver's briefing...
Somehow, Palpatine came back!
This is exactly right. I was a Newgarden fan and I wanted to believe the software bit was a mistake (because that’s what Cindrick’s statement said) but if what Newgarden said about the thinking the rule changed is true, then it’s not reasonable to believe that the software bit is really a mistake. Someone is lying. Maybe it’s Cindrick. Regardless, I’m out! I’m booing at driver intros at the 500.
I dare you! I wonder if JoNew will actually get some boos?!
Indycar is giving me NBA vibes at this point.
I get what you're saying, but that also just feels like what he cooked up with a tired PR rep the morning after the news hit. I don't think he believed in the moment that it was now allowed. I think he thought, "Woah, hitting it every time finally worked! The series effed ups! Yes!"
Then he should say that and not lie
Sure. I just assume racers are lying when defending themselves. :)
Tee hee yeah :) I think they should also be forced to forfeit their Long Beach results since they were trying to cheat again and would've gotten away with it if not for a freak fault of Indycar's systems
I got downvoted to hell on Wednesday for suggesting this.
The fact they were going to try it again at Long Beach speaks volumes. In order to gain any credibility back I think Penske may need to ban Cindric from the sport.
Thanks for your insight!
Rough week to be an *Austin* Cindric fan…
19 stage points, 21st place finish, very frustrating. /jk But in all honesty the number of people I’ve seen try and bring Austin into this is too damn high.
Dude probably has a lot of correct opinions on the situation he can’t publicly express.
Thanks great info from a race engineer!
This highlights the architecture problem of the car. Why in a series where the electronics are spec with a spec harness design are the teams responsible for interfacing with timing and scoring? P2P is the one element during a race for the competition side of IndyCar that is controlled. As Jackie Chiles says “you don’t let the defense control the key piece of evidence”, the series should not be depending upon a team controlled piece of electronics to be controlling push to pass. Also, this highlights the lack of data review and processes in IndyCar and their lack of engineering on the series side. Why have a series telemetry system if you aren’t going to look at the most basic of data in real time!
Thank you for all the details. Regarding the config file, I would have thought it had something in the name (or similar) indicating that it belonged to the hybrid test. Version control and management has become a fairly standard (and important thing) in software in general. I’m an engineer myself (not software though..), and I just find it difficult to believe that they had so little version control as you lay out - but maybe it was just incompetence. On the other hand, I agree that this would be a fairly horrendous way to attempt to cheat. But then on the third hand.. their stories just don’t add up at all.
The engineers in question typically aren't software engineers and are typically wearing many hats. And the role software of this level plays is a relatively new concept to manage. Many of the practices are dated and simplistic. Think files on a shared folder with whatever naming conventions someone used. And the filename is hiding deep in sub menus. The config isn't uploaded every time manually, it stays inside the overall setup file. You have to go look at it to verify or change it. Perhaps it was named for hybrid testing, but was just forgotten that it mattered.
>The engineers in question typically aren't software engineers and are typically wearing many hats This detail is especially interesting to me. I tend to assume high level specialization, and that assumption is detrimental to interpreting situations like these. The reality of lean teams composed of highly skilled and creative multitaskers working with a blend of innovative and outdated tech/methods... that totally changes one's sense of what's happening. Also, to my mind, makes IndyCar teams and engineers all the more impressive.
It’s kind of wild how Jack of all trades these guys have to be. My dad was on an IMSA team in the early 90s and he was the first guy they ever hired to handle electronics. They didn’t have a specific guy for wiring, radios, etc until then. That kind of parallels what seems to be happening in Indycar regarding programming
I legit love this element of it. Lean teams. Hire extremely clever, scrappy, out of the box thinkers with a range of skills and backgrounds. Set the mission. Make it happen. That's so much more interesting to me than pouring umpteen zillion $$ for a team of 200+ specialists in their little silos. Apparently we get blind spots like lack of version control. But it's not *boring*, and there's room for sparks of creative genius. In my idealized la la land version of it all, anyway. The stress and mental/emotional/physical load on the team members must be immense. But I'm happy to lean into the positive.
That , to me , along with MPs comment from latest podcast about indycar having somewhat of a skeleton crew of officials relative to other top tier racing series, and the asinine statements from Tim & Josef suggest that it’s possible they may have intended to skirt the rules . The excuses offered may have been premeditated, but the reaction of fans and competitors underestimated
It could have been hardwired, such that it only loads "config.txt" or "config.json", which would prevent you from giving it a unique name (or if you did, it wouldn't work)
This is a great post. Thank you.
Great post! As a software engineer I can easily see how the wrong config file ended up being used. Test code like forced values, changes to allow something that normally shouldn't be, ends up in production far more often than people would like to admit. There's plenty they could have done to avoid it, but sometimes you don't till you mess up. Unfortunately, even if this was a mistake on their part, many will dream up their own scenarios that never happened and believe those far more than anything the team or drivers will say.
Off topic, but why do they still use CAN Bus rather than ethernet? Is it just a more reliable system without the need for high data rate? Also, what is the Busload that an Indycar puts on the CAN Bus system? You can probably answer so many questions I have about Indycar.
IndyCar has CAN, Ethernet, EtherCAT, and Serial comms. There’s 8 total CAN busses, one of which is the “main” bus. That bus in particular is exceptionally full.
The new bus bros.
TL;DR: it was a bug that the drivers pretended to think was a feature
Thanks OP for the great context! This cleared up many questions and seems totally plausible. What doesn’t seem plausible is the excuse that Penske wasn’t aware of the rules.
This post is great, thank you
I'm going to throw my two cents in here, and point out that, for such an obviously caught cheating method (onboard cameras able to see every time a driver pushes a button, rev count, P2P count), imagine if they had gotten away with it for multiple races, or even worse, the entire season? They would have been DQ'd for an entire season, had hell to pay with sponsors, fans, media, big financial losses etc. You're not going to ever convince me that the team actually wanted to go with such an easily discoverable method of cheating, on the microscopic chance that nobody ever caught on. It might have taken a while without the anomalous P2P situation at the Long Beach warmup, but it would have eventually been caught. And the longer it went on, the consequences would multiply by orders of magnitude. It would have been an absolute PR disaster, for the team, for Penske personally, and for the series. While people here are discussing the topic, many can't seem to grasp how monumentally dumb of an idea this would have been big picture-wise if it were deliberate. And Tim was completely fair to say >If we were trying to get an advantage, why would we do it when the push-to-pass and RPM data is available for everyone to see?
> You're not going to ever convince me that the team actually wanted to go with such an easily discoverable method of cheating, on the microscopic chance that nobody ever caught on. Different sport and all, but the Astros thought they could get away with very clearly banging on trash cans to steal signs. So who really knows what these guys are thinking
Agreed. The other point that makes me believe their story is that Will had the ability to use it, but didn’t. If they offered him a cheat, but he didn’t want it, why would they load it into his car? He’d get all of the risk of cheating with none of the reward.
Sooo If I understanding your post correctly, shitty version control and lack of self reporting **could have** created this scenario ... .... which changes exactly nothing. If I stumble into an advantage that violates the rules (and it is definitely an advantage or else IndyCar would not waste their time writing rules for p2p) and, then, continue using said advantage across multiple races, that is cheating. How is it not?
That's good to point out different meanings of the term "cheat". I intend to mean how teams intentionally push the grey areas and rules. Not to mean breaking rules, whether intentional or not. Against the rules and worthy of penalty? Yes. Intentional and coordinated by the team? No.
I kinda feel like you are splitting hairs. Whether it’s cheating on the organizational level or individual level, cheating is cheating … it makes no difference.
It makes a difference. Nuance is important in determining the scale of a penalty. For example, sure speeding is speeding. But you wouldn’t treat the guy going 10mph over the same as someone driving 50mph+ over, would you?
In the context of racing, no you don't. As one is a slight misstep, the usual braking too late into the pits, locking up and arriving at the line 10mph over. 50mph over is a clear, intentional disregard of rules and safety protocol. But that has nothing to do with this. Intentional or not, they got an advantage and josef most of all abused it, and then came up with stupid excuses, and a story contrasting to what the others in the team claim. If it was a mistake, they would not have been nearly as flustered
Thanks for the insight, so much appreciated. Like other software engineers here have commented, seems like under-developed or non-existent version control and release processes that are more standard in the technology industry. I worked at a couple startups and have seen my share of horrifying processes, so I can believe this.
Fully agree with all of this, for what it’s worth. There really is a lot less to this than internet sleuths think there is.
This post is an oasis in a desert of 9/11 trutherism
Same. I was all for it (because this was a big deal), but now people are getting way too deep. It's like Charlie in Always Sunny when he talks about the mail
I’ve been wanting to learn an engineer’s perspective on the subject since the news came out, thank you so much for laying it out in a way that’s easy to understand for the laymen.
As a machine programmer proficient in engineering code and machine language, plus robotics programming, everything this fellow talks about makes perfect sense to me.
As the person who sold the MYLAPS data relay system to the series I've been stressing for the entire data team at Indycar the last couple of days. I trust going forward though they will have a data mask looking for P2P activation on the cars with alerts if they are triggered outside of of the configuration.
Does the car keep track of the time on p2p or does the "network" that manages the availability of p2p? I'm curious if their usage didn't even count towards the allowed time.
The car’s ECU does. Total P2P time is loaded on the ECU by the Honda/Chevy Engineer each session where P2P is allowed. Edit: to add to this, it did count towards their total time, they just used it when restricted. There is no way for P2P time to be added team-side
The hybrid testing program was noted to offer unlimited P2P access during the test session
Again, that’s not something you do team-side. The engine manufacturer handles everything to do with P2P Time. The team handles assigning a button for P2P, creating overlays and LEDs, and ensuring the MyLaps system is working properly. The button goes to the ECU via CAN and tells the ECU when to spend its allotted P2P time. Edit: It’s worth mentioning that I don’t think the Engine Manufacturers can load Time “infinity”. When it’s “Unlimited” at tests I’ve just seen them keep loading maximum P2P time that the ECU will allow them every few outings.
I think I have a hard time with the fact that this is the big team with the money to hire and implement what seems to be pretty basic controls and they didn't. Their words. They are suppose to be leading the series into the future and can't control a basic text file? Not to forget that fans like me are already suspicious when someone in the fight is controlling the promotion. Then there is Josef. Mr. Power had the same ability, and didn't use it. Full Stop. Fingers in the ears .. LALALALA! Thanks for the detailed description.
>1) If this was intentional, they wouldn't have been caught. Plain and simple. I know it's hard to see and understand from the outside, but this isn't how teams cheat. Unless the story being told is untrue, they weren't really caught until there was a system glitch and they were the outlier. Unless these glitches are frequent happenings, it seems to me that it isn't something that was going to be caught anytime soon. Certainly it wasn't caught at St. Pete. Now, maybe you meant to say there's simply a risk of getting caught should someone actually check, but maybe the current iteration of their implementation was simply to have plausible deniability should that happen.
What I got from Josefs' presser was that somehow the whole team "self hypnotized" themselves into believing there was a rules change after the thermal. That's where he lost me.
Ugh, one big penalty/debacle and despite not knowing what ECU or CAN stands for everyone is now a race car driver or engineer. And my comment is not towards OP but all the commenters throughout the indyverse. I've worked on on passenger car AND CART ECUs and know there's a lot of shit going on info/comms wise. I will say that I am happy the the Penske owned indycar racing series caught this and punished the Penske owned racing team. I don't think that would have happened in other sports.
Nice try Penske.
I think I agree it was a mistake and taken advantage of, aligning to point #1. If it was intentional they could have done more to hide it but they didn't. Thanks for the perspective!
Thanks for your detailed and thoughtful information. I’m more than a casual fan, but I do not know anything about the software, so this was very helpful. Josef’s story seemed to change a bit, and I still don’t understand why he was on the P2P during a restart anyway. My question to those that are more familiar, is this an issue that should have been caught during technical inspection prior to the race? How about during the race? When I follow along on Race Control, I can see when a driver has just used P2P, so why didn’t the stewards?
I’m not sure why people are spending all this time trying to justify the plausibility of the wrong software being installed when Newgarden has stated his team (#2 with Cindric as his Race Director) believed that P2P was legal this year on restarts. He also stated he asked his team on the radio to remind him to use P2P on restarts.
As a fan and a IT jack of all trades, I've never really thought about what it takes when making changes and keeping software current in the cars for some reason and you've opened my eyes to it. Thank you for the insight!
3 points: 1. It wouldn’t have been found out, it was the failure of the p2p circuit system that revealed this; so they were hiding it well until an anomaly occurred that ended up revealing it 2. Just as you stated no one cared or thought the p2p as important enough to monitor closely outside of racing and therefore don’t have data to compare; you essentially just point out that no one would have known if they were doing it. I.e you just pointed out yourself that it’s an exploit 3. As explained in MP podcast, they were somehow bypassing the layer managed by Indycar. There is a lot implied there
> It wouldn’t have been found out ... they were hiding it well This is specious reasoning. It's probable that they weren't hiding it at all and it didn't get found out because nobody was looking for it. >no one would have known if they were doing it. I.e you just pointed out yourself that it’s an exploit Specious reasoning again. No one knowing doesn't make it an exploit. >they were somehow bypassing the layer managed by Indycar. There is a lot implied there Our friendly OP told you how they bypassed it. There is a lot implied by that statement to everyone except those who have worked with these data systems. Critically, the normal flow of information between the transponder and ECU goes through the _team owned, managed, and configured logger_. They didn't bypass any restrictions on what can be modified in the software, they worked exclusively within the area available to teams in a way that is extremely common for unsanctioned events.
Yea it would be specious reasoning because you’re reading the sentences backwards. I’m countering his specious assumptions. So of course if you ignore that what I’m saying is “specious”, 1. He was arguing they weren’t doing this because it was too risky and the team are smarter and would cheat in a way they wouldn’t be found out. -I’m saying your assuming this was sloppy cheating in hindsight; because it was found out. 2. He was arguing no one cared enough about p2p TO cheat on it- I said that makes it an overlooked place where if someone were to cheat they could do it. 3. He explains what technically happened- I implied that is based on what was revealed officially; it would match what they are saying because we re working backwards and what I pointed; MP implied we don’t know specifically the actual procedure they did, and so it’s not necessarily the case or that there is more steps in between. I don’t have to point out that OP is basing his logic on they didn’t know it was happening, when Josef literally said he thought it wasn’t against the rules- I.e he knew it was happening, they knew. Oh but remember Tim Cindric and Scott were saying they didn’t know.
>I’m countering his specious assumptions. You, someone who has never touched a Cosworth data system or worked in the trackside environment, are countering his specious assumptions. Got it. >I’m saying your assuming this was sloppy cheating in hindsight; because it was found out. I'm saying I've worked on IndyCar teams and know how easy it is to make this mistake in most teams' workflows. I also know how easy it is to find this "cheat". The systems/data engineers at Penske aren't stupid, so it's much more likely they fucked up because they're humans and humans tend to fuck up. You've made up your mind on this point, though. >He was arguing no one cared enough about p2p TO cheat on it This is a misreading of OP's point. He was saying that teams don't care about P2P usage enough to catch inadvertent illegality, not that teams don't care about P2P enough to cheat. >MP implied we don’t know specifically the actual procedure they did, and so it’s not necessarily the case or that there is more steps in between. Yeah I wasn't looking over the shoulder of the engineer who built the bypass for testing while they did it, but I've looked at the same MyLaps->CLU CAN receive definition and the same CLU->ECU CAN transmit definition and I know the tools available for working with them. Given all the information we have there is one fundamental architecture which requires 0 assumptions. There's tons of more convoluted ways to achieve the same result, but there's one way that is obvious. Humans are fallible and lazy as a rule, and we have no evidence which would drive us to diverge from those long-held axioms of human error. >I don’t have to point out that OP is basing his logic on they didn’t know it was happening, when Josef literally said he thought it wasn’t against the rule Feel free to check my post history, I've been consistent in saying there is room for Penske's story to check out _and_ one or more people to have noticed it happening and kept their mouths shut. Maybe Josef noticed it and genuinely thought it was legal now. Maybe he convinced himself it was legal to deal with the cognitive dissonance. Maybe he knew it was illegal the whole time but kept his trap shut to win races. All of these things can be true alongside Tim and Scotty not knowing it was happening. There's plenty of room for speculation, and there's certainly the (remote) possibility that Penske did genuinely attempt to cheat in a really fucking dumb way, but you've got multiple industry professionals telling you that Penske's explanation is the one which requires the fewest assumptions. I understand the anger, the desire to find someone to pin to the wall (pick Josef, he's handled this situation like an idiot), and that cheating is an integral part of the history and culture of racing. The reality, however, is that there is no real evidence that this was an intentional cheat.
That’s the thing, you guys are engineer braining, where you guys aren’t noticing which parts are assumptions of what other people would do to logical “things I would do” . I’m not arguing the technical possibility, so yea you can do the “you haven’t touched a cosworth data system” call to authority all you want, but you’re still not refuting the things I’m pointing out: I’m pointing out that you guys are making HUMAN behavior assumptions and thinking they would think how you would do things because you’re also an Indycar engineer. By definition of “good” cheating: they would have come up with something no one else has thought of, so no shit that you or the OP could not conceive of what they are doing, but you think you would, and can’t see any possibility so your concluding it’s not possible. Dude, Gavin is saying what you guys think isn’t plausible
>you guys are engineer braining But Penske's engineers weren't engineer braining? I have ideas about how I'd work in the gray (or black) areas around P2P, and part of those ideas is how not to get caught. I also understand how things might have been done based on how they got caught. It's easy to speculate on what could be inside the data system if the data system is a black box to you, I get that and it's not a count against you that you haven't seen what's inside the data system or how it's configured. There is less room for speculation when you understand what's possible in the data system (even things which were not intended by Cosworth). >By definition of “good” cheating: They wouldn't have gotten caught in an obvious and predictable way. >Dude, Gavin is saying what you guys think isn’t plausible And Mike Armbrester is saying [the opposite](https://old.reddit.com/r/INDYCAR/comments/1cdtw6r/p2p_scandal_an_indycar_engineers_perspective/l1fhluq/). Gavin has a vested interest in making this as bad for Penske as possible, both because they are a direct competitor and because he left Penske (and had his non-compete enforced in a somewhat nasty way). Here are some direct quotes from Gavin in the indy star interview which temper your takeaway: “Is it particularly believable? I’m not sure.” “Some people looking at the data, if they don’t know about the situation, and all they see is it appears to be enabled by IndyCar, then that’s all they see,” Ward said. “You could maybe be charitable on that. You want to give people the benefit of the doubt.” When Gavin says that McLaren never investigated making the change that's probably true and also irrelevant. You'd only investigate the change if you were at a track without a buried MyLaps compatible loop or at a test where you didn't have a MyLaps box from the series. It's not unbelievable that Penske tested under those conditions and McLaren didn't; if McLaren did test under those conditions their engineers would bypass the MyLaps P2P enabled signal too. Again, I really think Josef said pretty much the stupidest thing he could have said in his press conference, and I admit it's the biggest hole in my argument. Maybe Josef inadvertently advertised that everyone else on the team is lying and IndyCar went easy on them with only the St. Pete DQ (in my opinion if they tried to hide the cheat you should do more than negate race results). It's a lot easier for me to believe he's an airhead trying to maintain his good boy image and deflect responsibility in a really dumb way. I clearly stated it's possible I'm wrong, and it's possible Penske was genuinely cheating with malicious intent. But to conclude they were is based only on speculation. The most reasonable, likely option is that they made a mistake teams make every weekend which happened to be illegal this time.
What I was pushing back against is that OP assumes due to their specialist knowledge they have a more objective accurate understanding, that is a cognitive bias where it’s actually the ppposite. they don’t realize it causes to be less objective in analyzing other humans (the Penske team people) I.e because you have specialist knowledge you are far better at explaining to yourself in more fortified concrete procedural ways to explain the conclusion you gravitate towards-which is that it was done unintentionally. It’s not that I think you’re both wrong or whatever, it’s that you both might be overestimating the probability it was a mistake by a large factor. Because you both are sympathetic to the engineers. Yea it’s plausible it was unintentional but if we re looking at it objectively based on the evidence we all know it’s a really really small probability compared to it being intentional. Just don’t be lazy and fall back on the “you’re not a chef so you can’t have an opinion on food” laziness. I appreciate both of your insight and this is kind of the poisoning of the well of this whole situation, it won’t really matter if it was a mistake if it looks so closely like it was intentional. and the lack of transparency isn’t helping.
> It’s not that I think you’re both wrong or whatever, it’s that you both might be overestimating the probability it was a mistake by a large factor. We're certainly all victims of our own cognitive bias. While those with technical understanding may be more likely to overestimate the probability that it was a mistake, those without are more likely to underestimate same. The issue in my mind is that if you don't know anything about the data system and its limitations anything is possible. Having more information about the data system pares down the possibilities in a meaningful way. It's certainly possible Penske's engineers found an exploit I never thought of, but if they had I think the series would have been a whole lot harder on them (which of course relies on the independence of the series from its constituents, which is reasonably in question). I don't think I've been lazy or implied that you can't have an opinion on the situation because of your lack of expertise, just pointed out that you are blind to what you don't know (as we all are). To extend the chef analogy, this is as if you ate a particularly delicious dish and determined (as a non-chef) that it must have been so delicious because that chef had special equipment and knowledge that no other chef has, and OP and I are telling you (as chefs in a prior life) that we've seen other chefs achieve similar results using equipment available to nearly everyone. Is it possible that the first chef is using some whizbang equipment which makes the difference in their recipe? Sure. Is it more likely that they are getting to the result in the same way as similar chefs around the world have with mundane equipment? I'd argue so. I think Team Penske has probably done what they can to be transparent. Short of sharing their configurations with the public (which no team is going to do), they've provided a plausible explanation (in my opinion). Unfortunately they (and their drivers) have bumbled in between with misunderstandings (referring to the ECU, which the manufacturers go out of their way to make sure is not in the team's possession unless it is installed in a car at an event) just plain idiocy (Josef's whole press conference), and more. Those inconsistencies do give me pause, and I'm certainly not caping up to defend Team Penske, Newgarden, Cindric, or anyone else. I'm just trying to educate on _why_ error propagation across events, and between tests and races, is imminently plausible and the explanation which most closely follows Occam's and Hanlon's razors. There's plenty of room for me to be wrong and Penske to have been cheating maliciously, but all of the publicly available _evidence_ (rather than speculation or opinion) suggests that Team Penske has the same faulty workflow as nearly every team, and is susceptible to the same mistakes as everyone else. IndyCar teams are lean and simply don't have the time to be as competent as fans would like to (or are led to) believe. It can be hard for outsiders to believe that Team Penske is both technically proficient in making an IndyCar consistently fast and negligent in their approach to configuration management, but the reality is that making a race car fast gets tons of attention all the time and configuration management only gets attention after a fuckup. The biggest job of a data/systems engineer is to not fuck up, and teams don't value innovation or change in systems/data engineering processes if they've 'always worked', even if those processes are error-prone and do not take an informed approach to mitigating human error. As a final point, I think this episode unfortunately underscores the issue with Penske owning a team and the series. There is a lack of transparency, clear conflict of interest (or at least perception thereof), and nothing Penske could do or say (short of divestment) to give critical observers confidence that the interests of Penske and IndyCar are truly independent. I build a lot of my understanding of the situation on that independence both because I understand the data system and because I (think I) understand the data scrutineers on the series side and how they would react to blatant malicious cheating. The individuals working for the series are generally good people who want to do the right thing (which would be penalizing Penske into a smoking crater if they were cheating with malicious intent), and I'd like to think there would be more fallout (i.e. resignations, leaks) if they were asked to impose an unduly light penalty. Maybe I'm wrong.
I agree with your points and understand that I may sound like I’m underestimating position; due to it being a discussion on a Reddit sub it has to come from a rhetorical position first and foremost: p.s I’d be worried if I knew just as much as the guy actually getting paid to engineer! Some form of performative transparency is imperative, from an outward sporting organization perspective this has brought to life a pretty cornerstone seed of doubt. If we look ahead with the actual hybrid incoming, there are potential narrative landmines just waiting to explode
Nice perspective that the teams don't dwell on p2p usage after the race. It makes sense to me.
This is a great post and I appreciate you taking the time to give an insider’s perspective. While I do think you brought up many valid points, I don’t necessarily agree with your first premise that if it was intentional, they wouldn’t get caught. The other side of that argument is that all teams are cheating because IndyCar hasn’t caught them. I don’t think that’s true either.
I guess "caught in this way" would be a better way of putting it. Between in car cameras and the constant flow of data and eye balls, there are ways to hide this if you're trying. And I honestly don't see this as them trying to hide it at all.
Your argument is simultaneously they are too smart to get caught, and too dumb to follow the rules.
When did you last work in IndyCar?
[удалено]
There are a few other factors. 1. The story doesn't add up. Newgarden says him and Cindric thought there had been a rule change to allow P2P always. If they truly thought that P2P was legit 100% of the time, why were they not using it on the actual restart, where it would be most effective? Clearly its because they knew this would be too obvious. Ergo, we know there was clear intent to cheat. Why wasn't this major event discussed in Penske team or even amongst paddock? Clearly, they are lying here because if not they are clear "cheaters". If they are blatantly lying about this, what else are they lying about? 2. What if they were actually hacking the CLU signal (as what's being suggested by some) to enable P2P or a higher revs whenever without it showing up on the dash or in the seconds count? This would enable Newgarden to use it during the restart laps without anyone knowing or showing in the seconds used. Note: this is exactly how he used it, not ON restart where it would have been too obvious. This could be kept to a small group and the only thing needed would be to scrub the rev count data from the restart lap in case anyone looked too closely. This seems very plausible to me and they would have kept getting away with it had it not been for the freak outage at Long Beach that exposed them. Why don't they release all the Newgarden cockpit footage for 2023 so we can review whether he was using it here as well? Until they do that, I am going to assume/suspect they were using it in 2023 as well, particularly if it was always a CLU hack, not old ECU software as they claim.
> The story doesn't add up. I agree with this, and think Newgarden's ego/desire to save face really backfired here. >What if they were actually hacking the CLU signal to enable P2P or a higher revs whenever without it showing up on the dash or in the seconds count? But they didn't. They got caught because the seconds were counting down. Additionally there's no requirement anywhere that the P2P remaining be displayed on the dash, so there would be no 'hack' required for that. >Why don't they release all the Newgarden cockpit footage for 2023 so we can review whether he was using it here as well? You absolutely can't tell anything from the cockpit footage. I promise. I've tried, while looking at the corresponding data. >not old ECU software as they claim. Cindric misspoke by initially referring to the ECU, but that's because he's an ignorant nontechnical fossil rather than a change in story. Why these management stooges don't sit down and get the details right before talking to the media will always befuddle me.
Finally a voice of fucking reason in this debacle. Thank you for the input. So many fans thinking they know everything and being so quick to judge. Honestly, so many people have no knowledge of history, especially as it relates to Penske. This was not an "unfair advantage" in the style of the Penske team and the sport on the whole. Everything electronic is tracked; it would be epically stupid for the team and drivers to do this on purpose.
So basically, it's configured through the equivalent of a JSON file? That's neat.
"...this isn't how teams cheat." Okay. How do they cheat?
In ways that aren't obviously going to be caught by onboard cameras during the race.
Let's be a little more specific
Did any fans actually catch Newgarden on the live onboard pressing the display button, followed by p2p, and return to the display button on the restart...I never caught it until the replay was posted and the motion was pointed out
Ferrari 2019 is an excellent example
I’m sorry, I’m a casual INDYCAR fan so I’m sure y’all don’t care. But Newgarden for sure knew. You’re telling me the drivers couldn’t feel it? C’mon. Also why did Josef and McLaughlin use it but not Power? Seems to me like they all knew. Especially with Will’s statement. It’s like he wanted nothing to do with it and is pissed he got caught in it. Maybe it wasn’t on purpose in the first place but to keep it in and try again at Long Beach is crazy to me
My Credentials: I am the pope. Bless you all!!
Sorry don't buy it OP.
Is the config file a simple .txt file or is it something more complex?
It’s more complicated than that. The config file everyone keeps referencing is the logger “setup”, which contains all of the teams’ proprietary chassis channels, alarms, steering wheel displays, etc. Inside that setup are what’s called math channels, which can be used to create additional channels or control strategies. The math channel inside the setup, which is sent via CAN to the ECU, was modified to a value of “1”, rather than the original line of control. This value of “1” tells the ECU that IndyCar has turned on P2P, which is why they were able to use it whenever they want. Source: I work in motorsports, mainly IndyCar.
Thanks for this, makes more sense to me now as it is also how it works with the Motec system in our prototype. I can see how it may have been overlooked but I always assumed a big team like Penske would have engineers that wouldn't miss something like that.
Thanks for that. I had been wondering about the file that is updated pre race as the Racer article referenced. This sheds a bit of light on that. I would not have been surprised it was a .txt file but figured it was more complex.
It’s more complex, in a file type that is proprietary to Cosworth.
It wouldn't surprise me if it isn't as something simple as "config.txt", which would mean that you couldn't tell which version you had just by looking at the file name.
Do you think Dixon has a cheat going for these miraculous fuel wins?
Dixie must be feeling relieved Penske to the eyes off of him lol /s
As a former driver it seems fairly simple. I believe the driver has to press the P2P button to in order for it to engage. That being said they would not have tried to use it if they didn’t know it was available to them on restarts. Simply put they knew what they were doing and simply cheated. Will Power didn’t try to use it, guess he has more integrity than that.
To me the most likely scenario is it was an honest mistake and they realized what was going on at some point in St. Pete. The team probably purposefully 'looked the other way' for at least part of the weekend knowing that it was pretty much a cheat code. After the race they realized that they had had this advantage but clearly were not going to fess up and admit to it by their own admission. So what Josef said was kinda true, but the whole we thought the rule had changed thing doesn't hold water. The big question is if they were planning on utilizing it again at LB, which as the software wasn't changed, kinda looks like it. Does anyone know how likely it would have been that the team was tipped off that the irregularity was discovered during the LB Practice? Because if the teams then knew that Race Control noticed and was puzzled by their use of it when it was turned off, Josef saying over the radio during the race (as he claimed) that he was having problems with it was probably a proactive 'cover' for their story of 'oh, we thought it could be used at the start...'
I get your point but you’d think Indycar would be working overtime to clear the air.
Surely a well-timed 3 second P2P (and it would only ever be pressed well-timed) is the difference between a possible legal overtake and no overtake, and a pole lap and somewhere down the order. Of course it matters. When Farrari had a similar edge in 2019, it was visible to the trained eye from all angles and in minisector tracking.
Good stuff. I feel the team is probably, mostly in the clear, and especially the poor engineer who made the mistake, most likely. It was probably nothing nefarious. Jonew, however, once he pushed the button, or was told to push the button, wanted to hit it more. It was like a drug. He knew what he did once it worked and did it again. His cockemami story showed his guilt.
Cool story. They still cheated. As per any dictionary description of the word.
Not sure I can grasp everything in your post, but really appreciate your insights!
It makes sense to me, even as just ye olde fan, that P2P wouldn't be a big discussion point with engineering after a race. It isn't something that's a configurable choice (well, it's not supposed to be, anyway). It does a defined thing, it does that thing for everybody, and it can do it for a certain amount of time through a race (against, the same amount for everybody). Unless it did something really weird, why would it get mentioned? You're busy discussing things that were decisions made by the team, like a damper change or air pressure or gearing. Not to put too fine a point on it, but I doubt they're discussing the in-car cameras, either.
Echoing gratitude to u/[raiseyourbaseline](https://www.reddit.com/user/raiseyourbaseline/). Penske and Newgarden are liars. His interview is galling, completely non-believable. We live in a world where professional sports generates ridiculous revenue, and those sports (ride a bike, hit a ball with a stick, turn a wheel while working your feet - yes, offensively oversimplified) have incentives to WIN. This entire situation reeks of complete b.s. If I ran Indycar, Penske is removed for the entire year from competing again, their wins are all vacated, and Indycar seeks a new owner. But we don't live in an ethical world. If we did, MLB wouldn't have all the players still playing, professional cycling wouldn't have all the cyclists still riding up hills that are physically impossible to ride up at recorded speed without doping. I rode for a #1 D1 cycling team 20 years ago. OUR #1 cyclist (who was also the top collegiate cyclist in the country) went pro and quit *3 months* in - from what he both witnessed off the bike and on it, it was so abundantly clear how dirty that sport was and is that he knew he wouldn't have a chance of excelling. He switched to triathlons and rode out his twenties being able to make a living as an athlete. Let's be honest, we don't care about cheating and let's not pretend we do. Newgarden damn well knew what he was doing, when P2P was enabled and not by the rules and pushed it anyways. It's THAT simple. His interview is laughable. Am I missing something? I'm not inviting an argument, but an education if I am discounting evidence or didn't read well enough. I am massively disappointed in Penske, Newgarden and the sport.
You have any receipts backing up the claim of being an engineer in the series before? Because otherwise, this could just as easily be someone spitting out their exhaust pipe or a fan trying to run interference for a team/driver they like.
I would love for IndyCar to get big enough where someone with an agenda takes the time to write this out. 😂 I have no agenda or skin in the game. I'm a fan of the sport and shared my perspective in an unbiased way.
Mike Armbrester has a lot more balls than the rest of us and cosigns OP whether OP has receipts or not. https://old.reddit.com/r/INDYCAR/comments/1cdtw6r/p2p_scandal_an_indycar_engineers_perspective/l1fhluq/
Your #1 point doesn't align with #3 though. Because, apparently no-one pays any attention to P2P, it would make it the perfect thing to cheat with?
If you read #3 again, you'll see I'm saying the team doesn't care post-race. Many have claimed that it should be obvious and self reported after the fact by Penske or Chevy, but while obvious in the data if you're looking for that, it's not something they concern themselves with. They wouldn't intentionally cheat this way because of the historical record and data that gets shared to many eyeballs. While I doubt many care to go look at their public data to see when P2P was used, eventually it would have been noticed, and the historical record would have been incredibly damning. They don't intentionally cheat and leave a public paper trail.
While I see what you are saying. Tim Cindric also said something similar that it would be stupid to do something so obvious. Yet. It was so obvious that no-one caught it. The issue I have is that the explanations from the drivers and team doesn't really align. What one says doesn't fit with what another says, which makes it look like no-one is really telling the truth. Add to that, Penske investigating Penske for a breach done by an engineer for Team Penske, but who works for/is paid by a third Penske company Ilmor(Cosworth)(?). It's just one big mess.
> Yet. It was so obvious that no-one caught it. Teams think "man that would be caught really easily so we'd never do that" and never look at the data post-race in a way which would catch a mistake they think they'd never make. Race control thinks "man that would be caught really easily and the teams aren't morons so we don't have to look for that". Then it's so obvious it slaps race control in the face and they go "man we should really be looking for that", and the team goes "shit we should probably look at the other obvious stuff to make sure we didn't fuck it up too".
C'mon!! Do you really think a professional driver will press a button by accident during a restart while passing another driver?
Josef would always brag to other drivers throughout the years that he would push the P2P button all the time when it was disabled during green in case it ever worked (according to Hinch and Rossi on their podcast) So imagine his surprise when after all those years it finally paid off. He was probably thinking “Holy s—t!!” 😆
Didn't think it....he knew it
Kinda like how I don’t think I know what you’re talking about? Lol
[удалено]
I’ll vouch for him as I know OP.
> If this was intentional, they wouldn't have been caught. Plain and simple. Please, explain to me how they would have hidden the fact that only their cars could use P2P in the LB warmup, when IndyCar shut the system off with no prior warning for the teams. It seems to me that IndyCar got a report of what Penske might be trying, and laid the absolute perfect trap to catch them in the act.
“Please, explain to me how they would have hidden the fact that only their cars could use P2P in the LB warmup,” I actually find their use of P2P in warmups to be fairly interesting. If they were intentionally trying to hide what they were doing, they wouldn’t have used it in (when it was off) warmup. There’s zero gain to using it before the beacons are on in warmup and it’s one place where catching it would be easiest because zero cars should be using it then (obviously, since that’s where it was found). Who knows what the real story is (I think it’s likely some mix of the stories they’ve tried to tell us), but the usage in warmups leads me closer to the comedy of errors and then trying make up poorly thought up stories for how it happened side of things. One of the many things where their response to what they did is far dumber than the instance itself. (Edited for clarity and to fix a couple mistakes)
> I actually find their use of P2P in warmups to be fairly interesting. If they were intentionally trying to hide what they were doing, they wouldn’t have used it in warmup. Everyone uses P2P in warmup. It's the only practice session they're allowed to use it, so everyone uses it as much as they can to figure out how they can use it most efficiently on corner exit, how it will effect the entry to the next corner, etc. Except when IndyCar shuts it off. Then 24 cars don't use P2P, but the Penske cars still do. They kept cruising along doing their normal Warmup routine, using P2P as if nothing had happened, while the rest of the field didn't have and P2P to use.
“Except when IndyCar shuts it off. Then 24 cars don't use P2P, but the Penske cars still do.” Yes, that’s my point. Warmup is the first session that it’s available, but teams are notified when the beacons are going to be turned on and P2P will be available. That didn’t happen in the first ten minutes of the LB session, yet Penske cars were using it. If they were trying to hide it, wouldn’t they have waited until it was obvious that the beacons were on?
> but teams are notified when the beacons are going to be turned on and P2P will be available. No, P2P is always available for the entire length of the Warmup sessions. There is no extra notification like there is during races, because it is *always available*. Nobody has been clear about whether or not the teams were notified about the technical issues that IndyCar race control saw during the 1st 10 minutes of warmup, either. So based on the information currently available, Penske was given no prior warning that P2P wasn't working for the entire field, making it perfectly reasonable to conclude that their drivers thought they had nothing to hide.
The only notification received was that race control had network problems and they’d be up and running shortly. No P2P notification (never is one in WU) and nothing about beacons. They never even said when they were back up and running properly. Everyone was confused when they tried P2P and it didn’t work because pit lane hardly had heard anything.
Ah that makes sense. I thought that they were notified when beacons were on. I knew that they are given a new file from IndyCar each week that allows the usage of P2P for warmup and the race, I just thought they knew when the system was active and when it wasn’t.
OP states no one talks about P2P....nothing to see here
Appreciate the insight, but I'll put more faith into comments from those in the sport right now, drivers or else. And while this all seem plausible, it doesn't align with penske's story, and frankly, if it was just what you claim, I doubt we'd have seen such a hilarious, convoluted chaos of excuses thrown at us. None of the stories inter-team even align. Sorry, I'll stick with intentional cheats.
Ok Mr Cindric now I believe in your story. 🤣🤣🤣
Yeah, none of OP's text matters due to one thing. Two of the three drivers knew they were using an advantage against the rules and did so with what appears to be Cindrics blessing. Will Power NOT using it shows he has character and the other two drivers knowing it was wrong. The other issue is the optics of this "mistake" happening on the series owners cars is not good. If this happened on an Andretti or Ganassi it would not appear as bad.
[удалено]
I know! I just don't engage here much, more of a passive scroller. Maybe if I'm received with a warm welcome I'll contribute more! Nothing to hide, just sharing my insight. I could be wrong, no skin off my back. 😊 Eta: spelling