Where software meets hardware
We are a software company. We are creating neural network applications. We are writing algorithms. We are definitely not planning to become a hardware production company. But we need to design our systems down to the lowest level, and there are no app stores running on aircraft. We can’t just buy servers or computer boards and run our algorithms on them, and here’s why:
Our systems are made to be airworthy. They work aboard an aircraft, which means the installed equipment should be: a) small in size and weight; b) physically robust and tolerant to high vibration and a wide range of temperatures and other environmental conditions; c) consume little electrical energy or, equivalently, not produce excessive heat, and d) be safe.
All this is nothing new to avionics developers. But the applications we are working on are based on processing very large amounts of data in a very short time, and no suitable commercial and airworthy hardware exists for that. Until now there was just no need for any avionics computational boards to perform at the level we require. Neural network and computer vision applications are very computationally demanding. Only a few years ago, the typical computing system was something like a 50 kg server box with a heavy cooling system.
Recent advancements in electronic chips delivered tiny processors able to host a powerful AI inside—just look at your smartphone—but even these don’t save the day for us. Airworthiness means the systems should be able to pass the rigorous certification, so their performance needs to be 100% predictable for every task, regardless of the external conditions, to have a very low failure rate, and still be very speedy.
Imagine that an exemplary aircraft is flying at a speed of 130 knots. Another plane moves toward it at a similar speed, making the approach velocity equal to 134 m/s. Due to the clouds, the intruder becomes visible only at a very close distance. Also, it is painted black and flying over dark terrain and is therefore not clearly discernible. In this case, a difference of 100 milliseconds in reaction, which would pass unnoticed by a smartphone user applying a cute filter on a cat photo, can mean the difference between detecting or not detecting an object.
Usually, high performance and reliability are considered orthogonal requirements for hardware. Graphics processors used for video games and scientific computing are high-performant, but not predictable, and what is known as 'embedded hardware' may be predictable, but is not performant. The general-purpose processor of the type manufactured by Intel or Apple in millions is a jack-of-all-trades: a general-purpose solution. It can do many things quite well, but it cannot do what we need to do very well. What we needed was something that does our thing, only our thing, and does it perfectly.
The same problem is addressed in the automotive industry (driver assistance functions and all other activities regarding self-driving cars), and they have developed their own solutions, which are already powerful and predictable. However, aviation is at least one level more rigorous for safety requirements.
So, we are designing our own circuit boards.
Tuning the grand piano for a gala concert: What our circuit design engineers do
We based our solution on a technology named FPGA, a cunning thing that allows you to create a particular architecture tailored to your algorithm (and only to it). Commercially available circuits, such as a CPU or GPU, can be very performant. GPUs excel at performing arithmetic operations for manipulation of computer graphics, but they are not predictable enough for our purposes. They are universal and give the best average performance for any application, but we need predictable execution for a known application. This is necessary for both our technical and certification requirements. As said by a character in Eliezer Yudkowsky’s book, an evil superhuman and the top-notch technical expert, “Even the greatest artifact can be defeated by a counter-artifact that is lesser, but specialized.”
FPGA stands for ‘field-programmable gate array,’ and this acronym nails its main feature: programmable hardware fabric. It’s a device consisting of the same logic elements as a conventional circuit, such as flip-flops, multiplexers, and other logic gates. What distinguishes an FPGA from a conventional circuit is that you can change the switching of signals between the logic elements. By doing so, you can fully customize the complex computational function performed by the device. In other words, you can program the hardware behavior.
When programming an FPGA, you define exactly how it should operate on a gate level, and you know, down to the nanosecond, what goes on at each tick of its clock. You can tailor your software to your hardware in the best way possible, and vice versa. You can also understand, optimize, document, and verify that it will always do its designated function to perfection.
Our FPGA designers can thus fine-tune the behavior of the circuit and ensure it only contains the functionality needed to fulfill our application requirements, which both improves power efficiency and reduces our certification effort. A single FPGA can also support multiple functions that would otherwise require separate components. We use this feature to save board space, and ultimately the weight and power, of our equipment.
Since the design of the full system is in our hands, we can interface the FPGA directly with our image sensors, enabling it to directly ingest video data. We built the FPGA to operate like clockwork with little interaction from the CPU, which results in both deterministic and low data-processing latency.
Why in-house?
Actually, the aerospace industry widely uses commercial off-the-shelf (COTS) FPGAs in avionics for real-time signal processing: radars and radio comms, networking equipment, fly-by-wire, engine control, augmented-reality, HUD, etc. These applications represent approximately 10% of the FPGA market revenue. However, the solutions suitable for most technological purposes are not up to the strict certification standards for safety-critical applications like flight control. To squeeze out maximum performance precisely for our purposes and ensure the certifiability of the final product, we need to design everything from the ground up to optimize every last bit, including the low-level software.
So, we have to develop a custom FPGA solution designed precisely for our purpose. This allows us to follow a strict, rigorous process of testing and documenting the code, an inevitable requirement for the eventual certification at the highest Design Assurance Level (DAL) we are aiming at.
This approach could be inefficient in a big corporate structure, but being a not-that-big and fast-adjusting company helps. “My ambition was to make something small, fast, and that nobody made before. After working on this for six months, I've already done what would take a lot longer in big corporations. I am not bound by traditional approaches,” said Richard Gordon, who joined our team to design our first FPGA application.
It takes more than one team
Niels Haandbaek, who leads the FPGA design work in Daedalean, explains the process: “We start the design process at the very highest level, with the product vision, ‘This is what we want the thing to do.’ After that, we work with the software team very closely on a component-by-component basis: we take their software algorithm, transfer it into the FPGA, and improve it even more. The software is a kind of blueprint of what we're doing, a model of what our neural network developers want to execute. When we translate it into something that works on the FPGA, we evaluate the performance and communicate the result back to the software team. Sometimes it’s ‘We can't actually do it in exactly the way that you have prescribed.’ Then we need to collaborate and iterate to modify their work and make it feasible to implement it on our side. It is a back-and-forth process. They come up with the new version and suggest, ‘Okay, let's try to do it this way,’ and then we go and evaluate it and see. ‘This will lead to these consequences for us’ – maybe we have to tweak things a little bit more, and it takes a couple of iterations, but now we are on the right track. So, we have the product vision, we have the software, and this, in turn, defines the requirements that we have to fulfill.”
Source: Own design
Okay, and what next? The FPGA designers don’t work with hardware. Their actual process is a lot like the software development process. They write in the special software tool a coded description of the configuration for the chip. Next, it’s the hardware team’s turn. They provide everything that this configuration needs to work on, create the actual tangible implementation, and come up with a hardware module that includes a main processor, line cards with FPGAs, cameras, and supporting electronics.
Our circuit design engineers are working with other teams from two sides: apart from working with the software team as described above, they work with the hardware team, providing them with the requirements and defining what the next piece of hardware shall look like. Everybody’s work influences the eventual result. We are working to create the product with the weight and power consumption twice reduced against the current prototype version. Everybody feels parental pride when the next version goes to testing, first in a lab setting, and then on a real board that goes flying.
Is there a special kind of adrenaline for engineers?
Many companies design or produce FPGA applications, but unlike most of them, we are our own clients. This is a fascinating approach for our engineers. As Niels Haandbaek said, “One thing that is very important to me is the fact that we're working on an actual product, we know the ultimate use in the cockpit. When you're working for a big corporation developing chips, you develop a technology, essentially. And then somebody else buys the technology and does cool things with it. You don't get to do the cool things, maybe only a tech demo. But at Daedalean, we get to do the cool things that get put into aircraft and actually fly around. For me, that's very motivating, but it's also challenging because you really have to make sure that you get to the end.”
There are few conflicting customer requirements, not much feature creep or need to support an extensive range of tasks. We get to build exactly what we need for a known application. Also, every detail of the solution must satisfy strict certification standards. This means we need to devise a lot more from scratch than usual, and every engineer can make their mark”.
Obviously, we are not the only ones struggling with the need to minimize size, weight, and power and using the new advancements in microchips for that. There has been a boom of “AI silicon” and “edge computing” for some years. Big players and new startups are working on this. An iPhone 13 weighs 174 g and contains computing power comparable to a laptop.
But: it is impossible to display 12 megapixels correctly on a phone screen, and an iPhone user only sees and works with a scaled-down version of an image. But: if putting an Instagram filter on your cat's photo takes 100 milliseconds longer, you're probably not going to notice that. But: the failure rate requirements for Siri are low.
Compared to that, our challenges are quite an adventure. It takes a particular type of character to wish to work under such conditions. Noam Gallmann, one of our machine learning hardware designers, adds: “I feel that I have a big impact on our final product, and this is very motivating. On the other hand, with great power comes great responsibility. I know that the price of a mistake is unaffordable, and this invigorates me as nothing else could."
