IATA Type B Bag Messages and Baggage Messaging Refresher

There is quite some movement in baggage handling and its associated messaging needs and requirements at the moment.  Though the IATA recommended practices RP 1745 and RP 1800 are around for quite a while, the IATA Resolution 753 (baggage tracking and custody) has to be implemented by June 2018 and the new BAG XML message standard is shaping up and will most likely released first time in 2017. Traditionally any handling of baggage requires a type-B message to be sent to the relevant parties. This is a push-based approach and due to the nature of type-B messages prone to errors (format) and accumulate costs by the distributing network operators and its transaction based charges. According to a IATA study/business case in the year 2012 26 million of bags have been mishandled, mostly for transfer bag handling and a good share of this is caused by missing or wrong messages.

This article is meant to provide an overview or general introduction, aka baggage messaging for starters. Baggage Handling is very complex process with dependencies and actors, including airlines, airport, handling agent and eventually the passenger and his baggage.

Main Systems involved in the process of baggage handling

DCS Departure Control System
BHS Baggage Handling System
BRS Baggage Reconciliation System

DCS
The Departure Control System is the operational backbone of every airline. It supports the check-in, baggage acceptance, boarding process and other related activities like load control, immigration.

BHS
The Baggage Handling System (usually owned by the airport) is a complex system of conveyor belts, chutes and bag drops that transports and buffers any checked-in luggage. It ensures that luggage that is checked-in, transferred or received from arriving flights is tracked, counted, scanned, screened and transported to the right bag chute or belt.

BRS
The Baggage Reconciliation System, usually used by the handling agent, helps to match passenger, bag, flight and container.

Traditional Type-B Messages for Baggage Handling (defined in IATA RP 1745)
RP 1745 defines the formats of the messages exchanged between the systems for automated baggage and passenger reconciliation, baggage sortation and other baggage services.

BTM Baggage Transfer Message
BSM Baggage Source Message
BPM Baggage Processed Message
BUM Baggage Unload Message
BNS Baggage Not Seen Message
BCM Baggage Control Message
BMM Baggage Manifest Message
BRQ Baggage Request

Baggage Tag Number or License Plate Code
A unique 10 digit number as reference for each piece of baggage, defined in IATA RP 740.
The bag tag number is part of the baggage messages.
According to resolution 751, effective June 1st 2013, the format contains only numbers.
Sample: 0220208212 (0-220-208212)

1 1 digit Leading digit 0
2 3 digit Airline code 220 Lufthansa
3 6 digit Bag number 208212

The printed barcode is a regular ITF-14 code, any smartphone can read the barcode. The number is also printed on the bag tag.

KLM is printing a “KL” in between but the barcode only contains numbers. “074” for KLM.

BTM
The Transfer Message contains bag information for the outbound carrier of incoming transfer passengers. Part of a through check-in transaction.

BSM
The Source Message is sent from the DCS to the baggage handling system upon checkin at the airport or bag drop.

BPM
The Processed Message is an status update sent locally, eg. baggage handling to carrier. BPM’s are often batched.

BUM
The Unload Message is the instruction to unload (or not to load) a specific bag, eg. no-show PAX at the gate.

BNS
The Not Seen Message contains bag info for baggage that could not been transported together with the passenger.

BCM
The Control Message serves secondary level information, such as
BAM Baggage Acknowledgement
FOM Flight Open
FMM Final Match
DBM Delete Baggage

BMM
The Baggage Manifest contains baggage details for down line stations.

BRQ
The Baggage Request asks for bag info from a baggage handling system.

Sample Message
A very simple sample of a transfer message

BTM
.V/1TOFRA
.I/LH123/14OCT/CPH
.F/LH234/14OCT/SIN
.N/0220588615021
.P/SMITH/JOHN   
.L/7FABC
ENDBTM
1 .V Version and suppl. data Transfer Station FRA (Frankfurt)
2 .I Inbound Flight Number and date CPH (Copenhagen)
3 .F Outbound Flight Number and date SIN (Singapore)
4 .N Baggage Details 10 digit bag tag id 0220588615021
5 .P Passenger Name John Smith
6 .L PNR Passenger Name Record 7FABC

Message Flow for interline flight

The below is rather simplistic view (sunshine scenario) of the messaging that happens around bag management for a 2 segment interline flight with through-check-in of bags.

Message Flow for interline flight
A passenger is flying on LH 401 from JFK to FRA and SQ 025 from FRA to SIN. An interline flight with baggage checked through Singapore.

Relevant Documentation or References

IATA RP 1745 Baggage Service Messages
IATA RP 1796
Baggage System Interface
IATA RP 1701f
Self Service Baggage Process
IATA RP 1800
Baggage Process Description for Self-Service Check-in
IATA RES 753
Baggage Tracking

Online References
Remark: Most of the IATA documents are not available freely and have to be purchased, here only links to public documents or pages.

IATA RES 753 https://www.iata.org/whatwedo/stb/Documents/baggage-tracking-res753.pdf
IATA BAG XML Initiative
http://www.iata.org/whatwedo/ops-infra/baggage/Pages/baggage-xml.aspx

Disclaimer: The information provided here might not be correct or complete. It is for educational purpose only. For reliable information please refer to the IATA manuals.

Airport AODB goes NoSQL (Part 1)

In previous blog posts I discussed ‘AODB and Big Data‘ and ‘AODB in the Cloud‘. As promised, in this third and largest part of the review, I will look at the NoSQL database approach, design a document datamodel, embed it into a MEAN stack and conclude in looking forward implementing an AODB in a Serverless Architecture using Microservices.

In this new series I will review the benefits and options of using a document-oriented database (NoSQL) and start a transition journey moving away from a relational database model to document database.

Robert Yarnall Richie, DeGolyer Library, Southern Methodist University

Lockheed 12A Electra Junior, Delta Air Lines at Dallas Airport in 1940 by Robert Yarnall Richie (DeGolyer Library, Southern Methodist University)

Before jumping into relational datamodel review and document design we shall have look at some industry initiatives and working groups that strive for standards with semantic models, business models,  information and data models and exchange formats and patterns. While a lot of airport systems have been developed years back in the absence of these models, but with best knowledge and common practice and experience in the field, we cannot ignore further the existence of emerging and established standards. For legacy systems is near to impossible to adopt the models at the core business implementation layer as products are usually designed around a datamodel which cannot be changed without a significant or even total redesign of the system. Here the approach is the adoption of the models at an integration and mapping layer. You can adopt eg. AIDX as messaging exchange format without having to use it as base of the product, though it creates additionally effort to create mappings. An additional challenge is certainly the number of models around because they were created by different organisations with different but often overlapping aviation domains in mind. I have to admit the organisations are cooperating and represented in the working groups to achieve a level of harmonization where possible. We look at IATA, ICAO, ACI, Eurocontrol, EUROCAE as lead organizations here.

Lets list the current models. This list is certainly not complete and only provides a brief overview. We can and should benefit from the availability of these models (most of them are freely accessible). A lot of standardization effort is going on at the moment, please note some models are to be considered as “work-in-progress”, some are quite advanced, major changes are not be expected and some are also due to submission to governing boards soon or in the process of it. Once the models, at least the exchange message formats, start materializing as official standard we will see them appearing in requirement and tender documents and soon to be out there to simplify system integration.

AIDX Aviation Information Data Exchange IATA XML Message Standard ***
AIDM Airline Industry Data Model IATA Model **
AIRM ATM Information Reference Model Eurocontrol Model ***
AIXM Aeronautical Information Exchange Model Eurocontrol Model ***
ACRIS Semantic Model ACI Model *
AMXM Aerodrome Mapping Exchange Model EUROCAE Model ***
FIXM Flight Information Exchange Model Model ***
WXXM Weather Information Exchange Model Eurocontrol Model ***
BAG XML
Baggage Message Exchange Eurocontrol XML Message Standard *

Status as of end 2016
*** official release available
* work in progress

In the context of AODB products I will look at the below models and message standards first, though all of them are important because there is no clear borderline in the heterogeneous IT landscape at airports, eg. it is a common request by users to see weather data being displayed in dashboards of an AODB despite weather is not a key entity. In the further blog entries, while establishing a new datamodel, we will also discuss the individual models. Some models focus more on ATM and less on airport related activities.

AIDX

Aviation Information Data Exchange is a XML messaging standard to allow information exchange between airlines, airports and other parties in the aviation community. It has been initially created in 2005 and was officially released in 2008, endorsed by IATA Recommended Practice 1797A. Being one of the old timer in this list it is already established and adopted by more than 100 entities. It comprises almost 100 distinct fields that cover most aspects of flight, aircraft and handling details, inclusive of A-CDM. The AIDX working group is governed by ASC (Airport Services Committee) and PADIS (Passenger and Airport Data Interchange Standards) board under the custody of PSC (Passenger Service Conference).

Please note that AIDX will be migrated into the AIDM (Airline Industry Data Model) which has a much broader scope than AIDX. We shall not ignore AIDX as it will be around for a long time in its raw format and we can expect the AIDM implementation would be quite close (to be discussed and confirmed).

The current release is 16.1. Please follow below links for schema and implementation guide.

 

AIDM and BAG XML

The Airline Industry Data Model (AIDM) has a very broad scope and encompass industry terminology, data definitions, relationships, business requirements.
Looking at an evolution from paper (eg. loadsheets ticket), teletype messages to EDIFACT, the emerging new standards as models and XML are the latest step in the evolution and promise to deliver a better consistency of definitions and data formats, as well an improved interoperability and faster system integration times.
AIDM is work-in-progress and give its nature and vast landscape it might be the continuous model for it, though confirmed standards will arise from it. One of the first implementations adopting the AIDM is the BAG XML initiative which improves bag handling related bag messaging, distribution and does away with the traditional type B messages (BTM, BSM, BPM, BUM, BNS, BCM, BMM, BRQ as per IATA RP 1745).
The documents are not public at this stage, only registered member can access the model which is build with Sparx Enterprise Architect.

In part 2 I will review a simplified relational database model for an AODB as starting point for our migration journey. Stay tuned.

Some reference websites and material you find below.

References Organizations

Eurocontrol https://www.eurocontrol.int
ACI Airports Council International http://www.aci.aero
IATA International Air Transport Association http://www.iata.org
ICAO International Civil Aviation Organization http://www.icao.int
EUROCAE European Organisation for Civil Aviation Equipment https://www.eurocae.net
RTCA Radio Technical Commission for Aeronautics http://www.rtca.org

References Standardization and models

ACRIS http://www.aci.aero/About-ACI/Priorities/Airport-IT/ACRIS
AIRM http://im.eurocontrol.int/wiki/index.php/ATM_Information_Reference_Model
https://www.eurocontrol.int/articles/airm-atm-information-reference-model
https://www.eurocontrol.int/sites/default/files/content/documents/sesar/8.1.3.d47-airm-primer-v4.1.0.pdf
AIDM http://www.iata.org/whatwedo/passenger/Pages/industry-data-model.aspx
AIDX http://www.iata.org/publications/Pages/info-data-exchange.aspx
BAG XML http://www.iata.org/whatwedo/ops-infra/baggage/Pages/baggage-xml.aspx
AIXM http://www.aixm.aero
https://ext.eurocontrol.int/aixmwiki_public/bin/view/Main/
WXXM http://www.wxxm.aero
FIXM https://www.fixm.aero
AMXM http://www.amxm.aero

( Model, implementation guidelines or schema available on website without registration.)

References Technology:

Disclaimer: This discussion, datamodel and application is for study purpose solely. It does not reflect or replicate any existing commercial product.

Visualization Use Case Part 2: Airline Arrival Delays in the US (Tableau)

After reviewing the flaws of the previous visualization of the DOT Airline performance data in part 1, I created an improved version with the same recordsets. It is a separate viz because the first version have some mistakes due to the number conversion during the csv import. I cleaned up, checked the data and used calculated fields to derive the sum of delays.

Airline Performance in the US 2015

Airline Performance in the US 2015

The basic concept is still the same, the matrix on the top left controls the dashboard, initially you see all data for 2015 combined, clicking into cells drills down.
I changed the barchart to stacked bars comparing total to delayed flight in one bar for each month.

Bar Chart

Bar Chart

I moved the split delay reasons into a separate bar chart and added a pie chart which reveals the main reason for delays (surprisingly weather and security have the smalles share!) The 2 lists are a Top 10 style lists highlighting the airports and airlines with the most delays.

Performance

Airport Performance

 

Performance

Airline Performance

 

How does the visualization transport information ? Let’s look at the strong and weak points of the second iteration.

+ The key information presentation is improved. We can see the viz is about delays.

 The dashboard starts to look a bit disorganized and the viewer eyes are moving around without a centre of attention.

+ The barchart now makes sense, you can compare total flights and delays.

– The detail delay reason over time does not create too much value as the distribution of reason is quite similar.

Conclusion: Spending more time on both data and visualizations improved the overall impact, though a bit cluttered.

Lets try to apply to some more tweaking..

Visualization Use Case Part 1: Airline Arrival Delays in the US (Tableau)

Going beyond sample datasets and basic visualizations I was looking for open data in my professional domain, the aviation and airport industry. Potential candidates for visualizations are connections, routes, flight plans, airport and airline performance. Performance is usually the comparison of scheduled operations vs. actual milestones. The delay of arriving or departure flights is not only affecting passengers and many parties inside and outside the airport community, but it is driving sentiments, perception and reputation and eventually costs money. This kind of data is not something operators like to release but thanks to the Freedom of Information Act (FOIA), a US Federal law, public gets access to all kind of statistics. From the US DOT (Department of Transportation) you can access and download a variety of datasets, one of them is the On-Time Arrival Performance of US airlines in the US and their delay causes since the year 2003 (link). You can filter by airline, airport and timeframe, review the summary on the DOT website or download the set as CSV for your own analysis. I downloaded the complete dataset for 2015, a 2,25 MB file with roughly 13.500 records.

Arrival Delays in Tableau

Arrival Delays in Tableau

 

Airline Delays in the US in 2015 by DOT

Airline Delays in the US in 2015 by DOT

 

It provides total arriving flights, cancelled and diverted flights, the delay count and total time by reason (weather, carrier, NAS, security, late aircraft) for each month-airport-airline combination for 14 carriers at 322 airports.

Airline Delays in the US in 2015 by DOT

Continue reading

Airports – Ready for the Cloud ?

Unlike airlines which are used to distributed operations and having systems like a reservation system hosted centrally at their hub (originating in the times of mainframe servers with access to this crucial part of their operations only via a remote connection), airports still tend to follow a much more traditional approach. Airport operations are local and and not geographically distributed like airlines, over decades they established local data-centres on-premise and created a mindset of full control only available with the server and IT services right in their basement. Along come big IT departments with teams of server-, network-, db-admins and support.

St. Albert at Dublin Airport, circa 1950 (CC by National LIbrary of Irland)

St. Albert at Dublin Airport, circa 1950 (CC by National Library of Ireland)

This paradigm is slowly changing, due to the fact airports need to cut cost and operate more efficiently. In parallel we can observe an attitude change at management levels, becoming more open to solutions which are outside of their physical control, they buy in the concept of SAAS, consuming a service on a subscription base with a well defined SLA and availability. This shift started with less crucial back-office systems, like Email-Server and document repositories, and now moving on towards more operation critical systems. Slow adopters or companies restricted by policies or governance issues start moving towards a private cloud, eventually cutting down on operations costs. Airports start to understand internet availability in the year 2015 reached a commodity level like water and electricity, they start to adopt even public cloud hosted services.
Zero tolerance systems like ATC or something less life critical like a FIDS system will remain certainly a local solution, but AODB’s are moving into the cloud. All the vendors jumped on the bandwagon and offer some kind of cloud solution, be in a private cloud offering (with the vendor) or even deployed to a public cloud. The potential in this approach is the opportunity to offer an AODB solution at a fraction of a price of traditional AODB projects. Deploying to a public cloud, without any local requirements other than an internet connection and a browser, a small airport can start using an AODB without any investment, maybe at a price as cheap as 3.000,- Euro monthly subscription. Assuming a smaller airport (less than 1 million PAX/year or something like 25..50 commercial flights a day plus GA) is operating with simple requirements (flight plan import and management, operational flight tracking,  billing, Type B and AFTN message interface).

To answer the questions: Yes, they are ready.
But it depends on the IT strategy of medium to big airports or the restricted budget and need of smaller airports.

Let’s see who is serving the long tail in the airport market !

Airport AODB and Big Data

There are 4 technology terms that almost every IT company in any vertical business picked up during the last few years: Cloud, Big Data, Mobile and Internet of Things. In this series I review some of these buzz words in the Airport IT business context.

Today I will review the question: Does AODB data qualify for Big Data ?

What is Big Data ? A term that everyone uses, from consumer, consultants, user to people at CxO level. All have it, want it, need it, do it. Everyone joins the crowd, though few really know what it is and how to apply it in one’s own context or if one actually has a reasonable use-case for Big Data. I would not attempt to explain Big Data in this article, rather pick key elements and try to apply them. Big Data is too big to swallow and it comes in so many use cases, technologies, brands, products, flavours and attributes from Amazon, Google, Facebook to SAP Hana, No SQL, Mongo DB, Map Reduce, Hadoop and NSA. You can google for the term Big Data, read the Wikipedia Page about it or read one of the hundreds of books about, I can recommend a few here that I read.

9390823839_a02d5e07fd_z

Terra Ceia Island Farms gladiolus being loaded onto a U.S. Airlines plane at the Sarasota Airport (by State Library and Archives of Florida)

Do not expect an explicit yes or no answer in the end, though we will do a Big Data compliance check for every paradigm, but still I like to be open to find use cases.

The 3 (+3*) key words for that usually used to define Big Data:

  • Volume
    An enormous amount of data created by human interaction, machines and sensors
  • Velocity
    A massive and continuous flow of data from various sources, usually in real-time, but not limited to.
  • Variety
    Data from many sources in structured and unstructured form
  • Veracity*
    Noise and abnormality in data.
  • Validity*
    Correctness and accuracy of the data
  • Volatility*
    Data relevance. Lifespan of the data in which data shall be considered for any analysis.

(* Only the first 3 V’s – Volume, Velocity and Variety – are the canonical Big Data attributes, the additional V’s show up sometimes in discussion or papers, but they basically apply to all data.)

Some basic facts about an AODB product. An Aiport Operational Database is an enterprise application with a closed and rather predictable user-base. Depending on the size of the airport and the community serving it we might have up to 200 concurrent users and 1000 active accounts. It is not consumer facing, it is not social media, no click-streams and no input of sensors.

Volume

We need to build a data scenario in a typical or average AODB setup to discuss the term volume. I will try to create a typical scenario in order to create a total number of records or attributes over a typical time-span. Please consider this is as a simplified sample, the figures might vary a lot, depending on various factors, like country and region, international or regional, hub, primary airport, etc.
Usually the airport size in publications or comparisons is derived from the number of PAX and or movements of aircrafts per year.
For some references please refer to the ACI website.

A midsized airport around 20 to 30 million PAX a year might have around 500 turnarounds, this will be 1000 movements (arrival and departure).

Lets assume every movement have ..
20 milestones (scheduled, estimated, actual timings, A-CDM milestones or proprietary timings,..). Each of these milestones gets 3 updates (in average!)
10 load attributes (number of pax, total, by class, infants, cargo and weights,..). Each of these attributes gets 2 updates (in average!)
20 network messages (IATA Type B, AFTN, CFMU,..) This can vary extremely depending on the system landscape.
25 various attributes (flight number, registration, tailnumber, flightplan ID, aircraft type, callsign, resources, connecting flights,..) Each of these attributes gets 2 updates (in average!)
This results in 150 attributes (inclusive of updates) per movement. Applied to 1.000 movements day, we will have
150.000 attributes per day,
4.500.000 attributes per month
27.000.000 attributes per season (6 months, one seasonal schedule)

This approach is conservative, it does not cover audit or system logging. It does not consider a situation where the AODB serves as central data repository (warehouse?), with data-feeds from other systems for permanent storage. In more complex environments I saw requirements to process and store 10.000.000 ground radar updates or 1.000.000 updates from the Building Management system a day.

Do 27 million attributes in 6 months qualify for big data volume ?
In this case I would say no, but taking into the account the option to store more than one season of data and maybe to cover more than one location in a multi-airport situation, maybe yes !

Velocity

Do 150.000 attributes a day qualify for big data velocity ?
Braking down to an average of 1,7 updates a second, rather not. It does not require an Big Data architecture to process this.
Compare with Twitter (not a fair comparison though): ~10.000 tweets a second.

Variety

First, we have almost no unstructured data. Once the AODB has been put in place and production there are hardly changes in the structure of the data. Unstructured data might come with free format messages or partial free format content.
The variety also depends on the complexity of the IT landscape and the number of interfaces, AODB’s often play the role of central system integration and we face lot of inbound data streams, but they usually come in an agreed format.

 

Thoughts on Big Data Analysis

One of the selling points of Big Data is the analytic you can apply to the vast amount of data to identify patterns, extract useful knowledge and business value from the data collected. This might help to improve your business strategies or processes or focus on certain areas of value, even predict future scenarios given certain repeating conditions. We can see definitely value added to the AODB context here, though lot of the data is given and can be adjusted only with limitations or not at all, eg. flight schedules provided by airlines (usually result from slot coordination procedures) and the airport physical resources (stands, gates, belts,..). The potential lies in the analytic of actual data, even the airport can’t necessary change schedules, but with patterns emerging from actual data vs. scheduled data (eg. delays in dependency of certain weather, season, etc.), we can optimize the resources. Analysing connecting flight info can help to improve turnaround ground times and avoid delays, detecting frequent aircraft changes can help to improve gate allocation an other scenarios.
And looking at the big picture, if we would be able to collect from a network or countrywide or on a level like Eurocontrol, Big Data analysis certainly will create more valuable insights and improve on-time performance.

Big Data Bookshelf

Big Data: Principles and Best Practices of Scalable Realtime Data Systems

Big Data For Dummies

Ethics of Big Data: Balancing Risk and Innovation

Data Science for Business: What you need to know about data mining and data-analytic thinking

Some assorted links

Airport AODB in the Cloud and Big Data

Working for more than 15 years with IT systems in airport operations I pretty much experienced most areas from development, testing, system integration, documentation, training and project management. I spend many years on and with a classic client-server based system, partially with legacy technology, and designed and built an AODB from the scratch in new technology and deployed it to the cloud. Two topics drew my attention particularly in the last few years, the “AODB in the Cloud” and “AODB and Big Data”, I like to review them in some short articles here. This talk is politics-free and does not refer to specific products or companies.

For the reader without airport operational background my elevator speech description for AODB. (Did you notice ? there is no entry in Wikipedia for AODB.)

AODB – Aiport Operational Database
An AODB system is usually the core IT system to support the airport ground operations, it integrates with various systems from the heterogeneous IT landscape found at the airport, compiling data from airlines flights schedules, flight plan management, communication between airlines and airports to building management systems, live ground movements and many more systems.
It serves as platform for CDM (Collaborative Decision Making) for the various parties forming the airport community, from airport operators, airlines, groundhandling agents to ATC (Air Traffic Control).
It handles seasonal and operational flights by providing both real-time and historical data, supports resource management for facilities and equipment, as well for staff, it is an information portal.

The AODB itself can be reduced to a simple 3-tier piece architecture, a database, business logic processing layer and a frontend.
It can be complemented by ESB (Enterprise Service Bus), BI (Business Intelligence) and other components.

Unlike consumer (software) products, this is rather a niche market, and AODB systems are offered only by a couple of companies. To name a few (in alphabetical order):
Amadeus, Arinc, T-Systems, IBS, Inform, Intersystems, ISO, Ultra,.. I leave it to the reader to find out more about these companies and their products.

The U.S. National Archives At Portland International Airport 05/1973

I like to discuss in the next few blog entries the following topics triggered by the technical and operational evolution AODB systems are going through.

Image: At Portland International Airport 05/1973 by The U.S. National Archives (cc)