So there are reports that each flight generates ~500GB worth of data.
1. How do airlines pull the data off after each flight?
2. What do they do with it then? (Dump to data servers? AWS? stack the drives in a room?
3. What type of changes has this large data set invoked on data analytics per aircrafT?

Airlines use the performance data to monitor the healthe of their airplanes and engines something like a 10 deg. C rise in cruise EGT could be spelling trouble in the engine. or a 10KT decrease in cruise speed could denote a problem with Flap Rig or excess trim to counteract mis-rigged ailerons,flaps or floating spoilers.

I’m not sure if the amount for data you are quoting is correct, it sounds too high. The aircraft automatically downloads the data over Wi-Fi at the gate, there are non public wifi networks installed at terminals for this purpose.

We store our own data, it is used for monitoring the aircraft systems and performance. Some of the data is shared with the relevant manufacturers.

500 GB seems like an almost unreachable figure unless the data includes photographs or video. That's literally a million pages of plain text.

https://www.telegraph.co.uk/education/s ... verything/
"To put this into perspective, a single Boeing 787 generates approximately 500GB of data per flight"

Maybe the data is generated but not stored or only stored if an issue is detected.
I'm curious as to whether there is any push for more real-time analytics vs post-flight. Especially with the new satellite links where you are able to obtain 50-100Mbps in flight

Acey wrote:

500 GB seems like an almost unreachable figure unless the data includes photographs or video. That's literally a million pages of plain text.

Not really. They're IoT heavy. Plenty of sensors, probably thousands if not tens. Assume they work average with 10 messages per second, and the message is average 100 of bytes, and you end up with a couple megabytes per second. Multiply by 10hrs sector (36000 secs), and you're getting pretty close, as for the estimate.

Cheers,
Adam

Myriad wrote:

https://www.telegraph.co.uk/education/stem-awards/power-systems/data-is-transforming-everything/
"To put this into perspective, a single Boeing 787 generates approximately 500GB of data per flight"

Maybe the data is generated but not stored or only stored if an issue is detected.
I'm curious as to whether there is any push for more real-time analytics vs post-flight. Especially with the new satellite links where you are able to obtain 50-100Mbps in flight

I saw some real time data tracking while at Boeing years ago. We watched a few subscribing 777s for a couple of minutes as a internal company demonstration of what was possible so we could pitch to the Navy for some of their planes (C-40Bs mostly). Reds came up for out of normal conditions so maintenance could be notified at the destination for corrective action if needed.

The Air France A330 that went down in the mid-Atlantic was sending failure messages as it went through its event. So the data pushing is happening.

I will also expand on my above answer. With the 787, a lot of equipment that did not provide diagnostic data in the past now does. Galley inserts (Coffee makers, ovens and microwaves) as well as passenger oxygen dispensers are hooked up to data busses that report their health back to the aircraft central computer. for reporting to maintenance..

Myriad wrote:

https://www.telegraph.co.uk/education/stem-awards/power-systems/data-is-transforming-everything/
"To put this into perspective, a single Boeing 787 generates approximately 500GB of data per flight"

Maybe the data is generated but not stored or only stored if an issue is detected.
I'm curious as to whether there is any push for more real-time analytics vs post-flight. Especially with the new satellite links where you are able to obtain 50-100Mbps in flight

I know the A350 for several flights can be downloaded on a USB. At the start when we got them the Airbus engineers who were there for the airline EIS phase would download this data from the maintenance computer next to the jumpseat.

The data on the aircraft is not stored in plain text it is on a database, and I am pretty sure the data when exported is in zip format as it then has some level of error checking with multiple files inside the zip.

zeke wrote:

The data on the aircraft is not stored in plain text it is on a database, and I am pretty sure the data when exported is in zip format as it then has some level of error checking with multiple files inside the zip.

It's probably something more advanced than that, we don't deal with critical data usually but are far more advanced with regard to data recovery from different databases. But it's behind aviation, so I'll just leave it here. I'd expect it's more-or-less what you said - db dump tool, single large file, compression tools (usually on-the-fly), verification, sometimes second copy if you consider data critical. Perhaps a bit more - but I expect more sophisticated tools rather, than change to the procedure.

Cheers,
Adam

gloom wrote:

It's probably something more advanced than that, we don't deal with critical data usually but are far more advanced with regard to data recovery from different databases. But it's behind aviation, so I'll just leave it here. I'd expect it's more-or-less what you said - db dump tool, single large file, compression tools (usually on-the-fly), verification, sometimes second copy if you consider data critical. Perhaps a bit more - but I expect more sophisticated tools rather, than change to the procedure.

I'd expect logging data to appear in (structured, syncable ) stream form.
Post processing might be into a relational data base structure.

WIederling wrote:

I'd expect logging data to appear in (structured, syncable ) stream form.
Post processing might be into a relational data base structure.

You're probably right, that's what I'd do as well. I was simply refererring to getting the data out of the plane, once it's in maintenance - where they get these. Internal processing is probably different story, but I have no knowledge how it's done on the plane, and this could be quite different from solutions I build - that is the railway. Some similarities do exist between loco and plane, but I wouldn't go far saying they're similar without knowledge it is so.

Cheers,
Adam

gloom wrote:

WIederling wrote:

I'd expect logging data to appear in (structured, syncable ) stream form.
Post processing might be into a relational data base structure.

You're probably right, that's what I'd do as well. I was simply refererring to getting the data out of the plane, once it's in maintenance - where they get these. Internal processing is probably different story, but I have no knowledge how it's done on the plane, and this could be quite different from solutions I build - that is the railway. Some similarities do exist between loco and plane, but I wouldn't go far saying they're similar without knowledge it is so.

Unix way of (local and remote) logging is about what I'd do. ( If you have networked environment.)

WIederling wrote:

I'd expect logging data to appear in (structured, syncable ) stream form.
Post processing might be into a relational data base structure.

The data I have seen exported is in xml format, with multiple xml files inside a zip file.

The network onboard is like a number of blade servers connected by a fast ADFX network.

This is a paper looking at getting data off the TAP A350s, it gives some sizes of data exported and the different networks the aircraft can use to do this. The majority of out data from the A350 is downloaded over the WiFi link at the terminal called gate link.

https://fenix.tecnico.ulisboa.pt/downlo ... stract.pdf

zeke wrote:

https://fenix.tecnico.ulisboa.pt/downlo ... stract.pdf

Interesting, thank you.