Late last year I became one of the first people in the world not employed by Mercedes-Benz to legally use the company’s Level 3 Drive Pilot autonomous system on a public road. German transport authorities had approved the system for use by regular drivers less than 24 hours earlier.
Fast forward to May 2022, and the Level 3 Drive Pilot system can now be ordered as an option in Germany on Mercedes-Benz S-Class and EQS models. It’s not cheap—Drive Pilot will cost the equivalent of $5300 on the S-Class, and $7900 on the EQS, where it must be combined with the $2600 Driver assistance Package—but the future never is.
Before the official on-sale date I spent some time in a Drive Pilot-equipped EQS on the traffic-choked autobahn that curls around the south and west of Berlin, Germany, in the jovial company of Jochen Haab, who heads the testing of advanced driver assistance systems at Mercedes-Benz.
Haab explained how Drive Pilot’s Level 3 algorithms had been updated since my last experience with the system.
Braking has been made to feel more natural when traffic cuts into the ‘safe space’ between the autonomously operating Mercedes and the vehicle ahead; the system still brakes the car rapidly if it senses a moving pedestrian in front, but cuts cars and trucks some slack before easing back to a safe following distance, just as a human driver would.
Haab said all the algorithms controlling acceleration, braking, and steering have been further tuned to ensure the car feels more natural when driving itself. That was a little harder to determine, given the natural smoothness and silence of the EQS’ electric powertrain compared with that of the well-used S-Class Drive Pilot mule I tried in Stuttgart, but the big electric Benz certainly wafted gracefully in the stop-and-go traffic.
Drive Pilot is easy to use. When the system senses that it is in what Haab and his engineering colleagues call its operational design domain (ODD)—in other words, the parameters under which it is allowed to operate—lights on switches in the steering wheel and at the top of the steering column glow white. Touch one of the buttons, and the system takes over driving the car, the lights on the steering wheel and column glowing green to confirm the handoff.
You can sit back and relax. As long as you’re on an autobahn, traveling at 37 mph or less in daylight on a dry road and the ambient temperature is 40 degrees Fahrenheit or higher, Drive Pilot will take care of everything, automatically steering and slowing and accelerating the car for as long as those conditions prevail, simultaneously calculating as many as 400 different trajectories in real time.
You can sit back, check your email, watch a video, or have a deep and meaningful conversation with your passengers.
You must keep an eye on those lights, though. If Drive Pilot senses it’s about to encounter conditions outside its ODD, the lights will flash red. You have 10 seconds to resume control of the car. If you don’t, the system will automatically bring it to a gentle halt with the hazard lights flashing and the doors unlocked, the latter to allow first responders access to the cabin if needed.
In truth, it’s hard not to notice the red lights, and, again, the handoff from Drive Pilot to driver is utterly seamless. Of course, you don’t have to wait for the car to decide to cede control: Simply move the steering wheel or touch the brakes and—as Captain Kirk would say—you have the conn.
-•-•-•-•-•-•-•-•-•-•-•-•-•-•-
Tunnel Vision: Why Mercedes Isn’t Following the Tesla Approach
Mercedes-Benz won’t copy Tesla and rely solely on cameras for autonomous driving capability. “Our philosophy is to have redundancies,” says Gregor Kugelmann, the senior development manager of the Drive Pilot system. “Each of Drive Pilot’s sensor systems have pros and cons. The combination of radar, lidar, and stereo camera technologies gives us the best and the safest sensor set needed for Level 3 autonomous driving.”
Though Kugelmann, or any of the Mercedes engineers working on Drive Pilot, won’t say it outright, the subtext here is clear: The Tesla strategy of relying only on cameras, with a little help from a front-facing radar unit, to enable a vehicle to safely drive autonomously at Level 3 is a strategy with a critical redundancy issue.
No matter how smart the AI system behind it, if your Level 3 autonomous system is relying on cameras and little else, there simply isn’t enough real-time data streaming into the car’s neural network to enable it to make informed driving decisions 100 percent of the time. Which, if, like Mercedes-Benz, you’re an automaker taking legal responsibility for any collisions that occur while your vehicle is operating autonomously at Level 3, is a hugely important issue.
Now, before any Tesla bros start kicking up a Twitter storm, below is a simple outline of the Drive Pilot system’s key sensors, and what Mercedes-Benz engineers say are their relative strengths and weaknesses. It’s clear cameras alone can’t cut it when it comes to delivering Level 3 autonomous capability.
It’s also worth noting that Drive Pilot also uses acoustic data. A sensor that measures noise from water spray in the front wheel well is used to determine how wet the road is, and the microphones in the cabin used for voice commands via the “Hey, Mercedes” prompt can pick up the sound of sirens from emergency vehicles approaching from behind.
In addition, the system uses data from the ultrasonic parking sensors, as well as the cameras in the dash that monitor driver fatigue and the usual data inputs from the powertrain, suspension, etc., and the whole system is underpinned by live high-definition digital mapping.
Long-Range Radar
How It Works
- Scans the environment with an electromagnetic wave, and records waves reflected by objects.
Pros
- Very good longitudinal range and velocity measurement of objects.
- Very good separation of moving objects.
- Sensor unit unaffected by dirt, dust, and fog.
Cons
- Lateral velocity of objects can only be roughly determined.
- Separability of stationary objects poor.
- No height measurement.
Long-Range Lidar
How It Works
- Scans the environment with a swiveling laser beam at various heights and records reflections from objects.
Pros
- Detailed mapping and separation of objects.
- Height measurement possible.
- High detection ranges for highly reflective objects.
Cons
- Sensor unit sensitive to dust, spray, and rain.
Stereo Multipurpose Camera
How It Works
- Optically detects objects
Pros
- Good object classification—e.g. a car, truck, motorcycle, pedestrian, etc.
- Good object separation.
Cons
- Poor longitudinal distance measurement.
- Sensitive to light conditions, especially backlighting.
——-
Q&A With Gregor Kugelmann, Mercedes Drive Pilot Senior Development Manager
MT: How long has Mercedes-Benz been working on Drive Pilot, and what was the original goal of the program?
GK: We started the early development in 2015. The target was to have a feature that would give our customers time to do things in the car other than driving, like email, browsing the internet, watching videos.
MT: Has the change from Level 2 capability to Level 3 capability primarily been driven by sensor development?
GK: Yes. We needed a third sensor at the front of the car—a special lidar we developed together with Valeo—combined with a live HD map that we developed together with Here Technologies, a digital mapping company, to be able to locate the car precisely, to the centimeter, where it is on the road.
MT: How would you describe the digital driver that drives a Mercedes-Benz at Level 3?
GK: Very good question. We had a long discussion as to how good, how passive, or how effective should this driver be. We decided it should not be an average human driver, but also not a Formula 1 driver like Lewis Hamilton. We call it an advanced driver, better than 70 to 80 percent of normal drivers.
MT: Where did the data points or the modeling for that come from? Was it observing real-world drivers, real-world feedback? Was it expert drivers within Mercedes-Benz?
GK: Both. We did intensive field study as well as scientific study and research. We did a lot of testing in our driving simulator with different people, from inside and outside the company, simulating different traffic situations involving up to 14 different cars.
MT: How much can the Drive Pilot’s existing Level 3 capability be expanded, with the existing sensor set and software set that you have in the car?
GK: We have a small-step approach. The first goal is to have an extremely safe and robust system on the road right now. But I would like it to operate earlier in the morning and later in the evening, in the twilight, because those are typical peak [rush-hour] traffic times. Maybe we can also add some slight wet road capability and increase the speed a little bit if the law allows it. These are the small steps that we will be working on in the next months and years.
MT: What do you need to do to make the next step in Level 3, which would involve autonomous operation at higher speeds with lane-change capability.
GK: For the lane change, which will be mandatory if you want to travel at more than 37 mph, we need to add some long-range radars at the rear of the car, to be sure the system can sense overtaking vehicles on both sides of the car before you go into a really safe lane change.
MT: Level 4 autonomy is a major step. How difficult is that to do?
GK: The big difference in going from Level 3 to Level 4 is the human driver is not the fallback. In Level 4 the car has to handle everything on its own. This really is another dimension.
MT: But wouldn’t a full Level 4 Mercedes make commercial sense with vehicles such as V-Class vans, which are used as taxis or delivery vehicles?
GK: If you talk about the V-Class, we are definitely doing some things in China. And bringing parcels from point A to point B, on the same route every day on the same time, might be an option for a Level 4 system.
MT: Technically, you already have some Level 4 capability in your cars, don’t you?
GK: Yes. We are about to get official certification for our Level 4 AVP system—the automated valet parking system—in Germany.
MT: There seems to be two pathways to get to Level 4. One is by evolving existing Advance Driver Assistance Systems (ADAS) used by existing automakers, and the other involves building a perfect digital driver from scratch, like Waymo and others are trying to do. Is there a race on?
GK: There is definitely a race on, but I think the philosophy between the two approaches is really different. We still stick to our goal of selling cars to our customers that offer autonomous technology to give them time back in the car to do things other than drive when they want or need it. Building a self-driving car and getting rid of the driver is another approach, but it’s not ours.
Keyword: Mercedes-Benz Drive Pilot Level 3 Autonomous First "Drive": We Try a World's First Driverless System