Organization

Tuning Do's and Dont's

Introduction

If you're new to tuning, it's good to have a guideline to how you approach this process. That's why we have this document, to help you get in the right mindset to make your goals with your car realized with the reliability you want. These are based on our own experiences, or own successes, failures, and hard lessons learned. We hope that we learned some of these things that hard way so that you don't have to!

The Dunning-Kruger Effect

To understand the premise behind the Do's and Don'ts, it's good to understand and reflect on the Dunning-Kruger Effect. This is a cognitive bias that was first described by David Dunning and Justin Kruger in 1999. It describes how competence (in our case, background in the physics and experience behind-the-wheel tuning engines) relates to confidence.

With low competence (starting out in tuning), it's possible to have an erroneous, over-estimation of your skills which leads to thinking you know more (are more competent) than you do/are. The Dunning-Kruger effect tells that we are more likely to overestimate our abilities as we are freshly starting to learn how to tune our cars. In many disciplines, this doesn't carry huge consequences. Unfortunately for tuning cars, though, the consequences of this overconfidence can be as serious and expensive as a hole in your engine.

In essence: this entire page is attempting to help you avoid being in the upper-left region of this chart.

image.png

Do's and Dont's

This list isn't exhaustive, and examples provided aren't anywhere near complete; make sure you do your own research to expand on areas or ask us questions in our community Discord where you are interested in learning more. If you find yourself doing things in the "Don't Be This Guy" category, understand that this is only our assimilated opinions. We humbly provide these contrasts as one more guide of many available for ourselves and for our community members to grow as more objective, data-driven tuners.

Aspect Don't Be This Guy Be This Guy
Limits
  • Don't start making changes to your tune without identifying your limits, both with the car and with your own background knowledge.
  • Don't assume what the limits might be, even if you feel like you have an intuition based on a prior project.
  • Don't ignore your limits, because the laws of nature aren't ignoring you!
  • Ask for help and identify what the limits are to the vehicle you're working with. Here are some examples:
    • What timing advance is reasonable for your fuel?
    • What boost pressure yields the greatest efficiency on this turbocharger? What boost pressure just "blows hot air"?
  • Everything in life has a limit. Even drinking too much water can be fatal (it's even got a name). So does your car, and reflexively, every aspect of your tune has some sort of a limit. Know them.
  • Stick to these limits, if that's your plan. Understand that if you don't know, it's better to stop and ask for information or help from our community before continuing.
Risks
  • Don't ignore risks or underestimate the possibilities for critical component damage (i.e. engine).
  • Your risks are derived from your limits; don't fall into a false sense of security by disregarding limits.
  • Don't make assumptions about your luck. Luck won't make your tune a reliable one.
  • Don't let a fear of your risks limit yourself from realizing your goals. A proper risk assessment should give you the courage to approach your realistic goals with confidence.
  • Know your risks as you learn your limits. Here are some example questions to ask yourself:
    • What are you risking by increasing the boost too much?
    • What outcomes can you anticipate if you advance ignition timing too much?
    • What are the possible risks of running to lean or rich?
  • Realize that your risks are always present whether you want them to be or not, and physics doesn't care about your luck or your gut feeling.
  • Form a personal assessment. Compare it with your capabilities, both financially and your part reinstallation options. Can you afford to risk an engine? How much uncertainty in reliability are you willing to accept by inching closer to or even past a limit?
  • Be confident in how your approach to tuning reduces the risks you're trying to avoid, and base that off of examples. Who else has accomplished your goals? How did they do it?
  • You can never be too safe; get a second opinion. Are you and others who are ahead of you in the journey comfortable with your approach?
Goals
  • Don't be the tuner without a goal.
  • Don't keep adding boost, timing, etc. just because it feels good and you're waiting to see if it'll be fast enough this time. That itch might not go away until it's too late...
  • Don't end up looking back and wish your goals were more realistic! As is often said, "my car felt the best it ever had before it threw a rod!"
  • Don't "scope creep" (break beyond your goals) if you haven't mentally revisited your limits and risks associated with your new goals and are, once again, okay with the risks.
  • Set your own goals. Have an end result in mind.
    1. How much X (horsepower, boost, etc.) do I want?
    2. How do I want to use this tune? Do I prefer to joyride my car, auto-cross, rally, or do I prefer to run the quarter mile?
    3. Now, how much X (horsepower, boost, etc.) do I need to enjoy that?
    4. Now, what goals do I need to set so I end up with a tune that makes my car run the way I really want my car to run?
  • Be realistic. Scale back your goals to reasonable ones that are within your comfort zone and don't push the boundaries of what limits you can't afford to exceed.
  • Once set, stick to your goals until you've reached them and you determine that it's time to set new ones.
Learning
  • Don't set yourself up to learn things the hard way. This will be a lot easier to avoid if you learn your limits, assess your risks, define your goals from those, and stick to them.
  • Don't blindly throw yourself at a problem without having done at least some research beforehand into possible root causes.
  • Don't overestimate your own understanding.
  • Don't forget that every platform is different, and a prior platform almost never affords you everything you need to learn about a new platform, even if you've tuned other platforms for a decade.
  • Recognize when something has gone wrong, and do your best to learn from your mistakes. Something unexpected in your data is an opportunity to expand your understanding and learn from the experience.
  • Have some humility, and do your best to be open to other's advice.
  • Be fully informed prior to your map changes; learn anything and everything you can before making adjustments, even seemingly small ones.
  • Furthermore, understand what your map changes mean in the physical world. What is timing advance, really? What is the wastegate's job? How are these things controlled?
  • Learn your sensors. What sensors does your vehicle provide? How do they work? What do the reported values mean? How and when can your sensors report values to you incorrectly?
  • Learn and be comfortable with stock reported values. What does the OEM tune command? Why is it commanding it?
Process
  • Don't tune without a process in mind.
  • Don't disregard the importance of root cause identification.
  • Don't disregard your process and start trying random things if feedback from your car is suggesting your approach was wrong.
  • Making a change to your tune to realize one of your goals? Have a process.
  • When you have a problem, always identify the root cause with the evidence and data to irrefutably back it up.
  • Get a second opinion on your approach to reach your goal or solve your problem. Is there a better way to dial in that wastegate map?
  • If things still aren't working to produce the result you want, revisit your process. How could it be improved? What could you be missing? What do you need to learn?
Data vs. Assumptions
  • Don't make assumptions.
  • Don't disregard your data in your logs. It's there to guide you.
  • Don't only log what parameters you think you need to log. Don't only record logs with parameters that help you confirm your theory, record logs with parameters that help disprove any other possibilities.
  • Don't fall victim to confirmation bias, even if your data has been teasing that you are right about your intuition; it could still be misguided.
  • Don't make a guess without data to back it up. For example, if you weren't logging when that problem happened, don't start making adjustments until you catch it again with data to prove your theory.
  • Get comfortable reading and understanding data. Don't make assumptions about what you think is going on. Instead, get data so clear that it's good enough to convince others.
  • Log anything you can that might be relevant to your situation. It doesn't hurt to have more data available to correlate.
  • Doubt your own preclusions unless you can verify them completely with reliable sensor data.
  • Understand your knowns and unknowns:
    • What do you know? This should be derived from your sensor data, not driver's seat feedback from the vehicle.
    • What knowledge gaps do you have that you're fully aware of? These are called known-unknowns. This could be, for example, a possible condition that you don't have a sensor available to truly characterize, or even a parameter you have available, but didn't datalog in that last run.
    • What could you theorize you might be missing altogether? These are your unknown-unknowns, or things that you don't know, that you don't know. A broader, deeper understanding of the physics involved helps eliminate these.
Iterate
  • Don't make giant adjustments to skip over steps that might seem insignificant to you, even if it's not your first time following those steps.
  • Don't assume that you're ready to iterate again towards your goal if you haven't developed a broad confidence in your current tune.
  • Don't change so much at once that you introduce too many variable to any possible problems that could arise from those changes.
  • Iterate. For example, don't make one big adjustment, divide it into 3 small adjustments with a test drive for each. Build confidence in your understanding by observing incremental success at each iteration.
  • Prefer small adjustments, even if you have confidence that larger adjustments will surely be fine. This is, unfortunately, a hugely common failure point for most beginners and suggests lack of understanding, poor assumptions, and finally is a good inflection point to revisit their risk assessment.
  • Prefer changing one area at a time, or minimize the change to multiple areas at once. If you change too much, you risk introducing too many variables that might make diagnosing problems stemming from those changes very time consuming.
  • Save often and keep a history of previous iterations. If you hit a problem, sometimes the easiest thing is to start from yesterday's map and step through the changes one-by-one, retesting as you go.
Patience
  • Don't expect to get to "X" horsepower at the end of the week or month. Be realistic with your capabilities and your experience.
  • Don't let frustration with your process distract you from your limits or enable you to disregard your risks.
  • Don't rush yourself by not setting aside enough time to tune, to deny yourself the ability to produce quality results.
  • Be patient. The more time you spend tuning your car, the better your results will get.
  • If you're waiting on advice, don't lose your patience and start trying things. You asked for advice for a reason, right? Maybe ask others if a reply is taking too long from one resource.
  • If you've been suggested to slow down, slow down.
  • If you've been told what you're doing is dangerous and you should stop and rethink your process, stop and rethink your process.
  • Know what steps you'll take if your process doesn't work the way you expected it to.
    • For example: if you run out of ideas out on the street, will you be comfortable accepting that you need to do more research and head home, or will you get impatient, make unplanned changes, and possibly make things worse as a result?

Closed Beta Introduction & Overview

Introduction

Atlas is a open tuning project, provided for free to the Subaru community as a resource to recalibrate (tune) modern Subarus such as the VA and VB WRX.  Even though we are a wide open door and do not impose any paywalls, licensing, or tune restrictions (beyond emissions tampering) on any number of vehicles you wish to tune, we do have certain tiers of releasing that does mean new features are released to a select group of individuals before the general public.

Although our public releases are typically a quarter-by-quarter phenomenon, truthfully Atlas is evolving every single day with enormous features and changes under the hood on a weekly basis. Our codebase is architected to enable us to move extremely quickly, unusually so for the tuning industry, and we simply cannot overwhelm the public with quickly evolving releases. But, we still need a solution to get our changes tested.

Closed Beta and the Fast Ring

Our Closed Beta is a small group of volunteers (roughly a dozen) who are hand-selected by our development team to field suggestions, mold our releases, and test new changes.

The Closed Beta group comprises our fast ring, which is the group that receives bleeding-edge development releases as they are made available from our development environment for Atlas. Due to this tight integration with our extremely high-paced development environment, it is not uncommon for 2 or more patches to be made available to our Closed Beta testers in a single day. Beta testers are regularly reinstalling Atlas, upgrading projects, and providing feedback, sometimes doing all of this on an hour-by-hour basis.

Support

We take this group and our commitment to its members ability to flash very seriously. For example. if an ECU ends up bricked due to our code (which is exceedingly rare), we have spare ECUs ready-to-go in storage in multiple team member's hands that we can overnight ship, free of charge, to our closed beta testers. We also typically are able to provide direct, same-day support to our beta testers using software like TeamViewer, and we do this for free. If you can't flash and tune, that is a stop-the-world event for us, halts all development, and your car immediately becomes our top priority. If you are experiencing mechanical, electrical, or tune issues, we expect our small community to swarm to each-other's aid.

Culture

Our ultimate goal is maximum efficiency, ease-of-use, and overall product satisfaction from our end-users, so our Closed Beta groups and every single member is expected to be a part in setting the standard for that. Our Closed Beta members, ideally, become key members in our community that help guide new users using Atlas due to their extensive experience and background with the software. But, in reality we're just a bunch of car enthusiasts that hang out and - for the most part - just have fun.

There is a Closed Beta group for each generation of vehicle we support, and they are separated from each-other so as to help reduce noise and clutter. We do this on purpose as it's best for discussion between beta groups to happen in the public, so the entire community can gain from this as well.

New Additions and Requests to Join

We also recommend reading through and familiarizing with our Tuning Do's and Dont's. We prefer if you try to be the "guy" on the right in green, but we're not going to force you to.

Due to the high bar for support excellence and the time commitment it involves, and being a tight-knit community with its own culture, we are meticulous about who we add into these groups.

Expectations

Each of these expectations is paramount, and sadly we have had to make the tough decision to remove closed beta members for not adhering to expectations, so take these into serious consideration if we're asking you to join a Closed Beta group: