Airport AODB goes NoSQL (Part 2)

airport

Earlier I embarked on the journey to create an AODB based on a NoSQL datamodel, moving away from a relational model and discuss its benefits. As a quick refresher about What’s an AODB ? for the new reader, the elevator-pitch style version describing an AODB:

AODB – Aiport Operational Database
An AODB system is one of the core IT systems to support the airport ground operations, it integrates with various systems in the heterogeneous airport IT landscape by processing data from airline seasonal flights schedules, flight plan and slot management, ground movement from Radar, air movement from ATC, and other sources. It serves as CDM (Collaborative Decision Making) platform for the various parties and stakeholders forming the airport community, from airport operators, airlines, groundhandling agents, authorities to ATC (Air Traffic Control) and others.
It handles seasonal and operational flights by providing planning, real-time and historical data, supports resource management for facilities, equipment and human resource and feeding information to public via FIDS and other external links. The below diagram shows an exemplary common orchestration of systems where the AODB is embedded at the core.

airport_systems_20181030

Now lets have a look at the typical data layout and relation of flight data entities and attributes. These are the common business entities and a relational model is the traditional approach to design it. We need to apply a rather high level of normalization to avoid redundant data, but the relations (typical 1:N) across the model have an impact on the performance of the DB. This can be counterbalanced by tuning, indexing and more powerful hardware underneath. Building SQL statements with joins across several tables becomes challenging (hard to create) and might cause inefficient reads of tables (full table scan). In comparison with a NoSQL design we have a document approach, one document (like a index card in the analog world) would contain all relevant data (ignoring the redundancy problem for now).
At the end of this exercise we have to ask the question: Is NoSQL the right tool for an AODB ? (We will revisit this question later on)

objects_20181030b

I like to elaborate the redundancy problem on one particular case:
A flight is operated with a specific aircraft (registration, tail number) on a certain date. The related information (AC Type, seats, owner, lease, etc) we retrieve from the relational table (containing all aircrafts in the system,) quite the standard scenario. The problem starts when we keep operational data long term (years) for auditing/research/statistical purpose. It is quite common registrations get transferred due to sale or scrapping of aircraft (find a sample here). Using the relational model with an aircraft registration table that only carries current registrations we would end up looking at the wrong information for a historical flight that operated on the previous aircraft with the same registration. A solution would be introducing the concept of validity for certain entities ,which again adds to the complexity.

The main problem is not solved, we should not replicate or mimic a relational model with NoSQL. Keeping the data redundant will increase the data volume but we would have one document with all relevant information. One usecase which is appealing for the document approach is creating a final snapshot of the flight in an archive like repository. The design question we have to answer, what data or details of the operations lifecyle (schedule, planning, operation, post-operation) we want to keep in the flight “document” ?

As an academic exercise, lets get started and create the most basic (primitive) version of a flight document in JSON format and look at all its weakness to start to evolve to improved versions of it.

{
  "flight": "AA123D",
  "org": "AKA",
  "des": "FRA",
  "service": "J",
  "actype": "A350",
  "position": "Z19",
  "gate": "A5",
  "baggagebelt": "09",
  "scheddep": "2017-11-23T19:35:00.000Z",
  "schedarr": "2016-11-24T13:15:00.000Z",
  "estimatearr": "2016-11-24T13:55:00.000Z",
  "estimatedep": "2016-11-23T19:39:00.000Z",
  "onblock": "2016-11-23T13:35:00.000Z",
  "offblock": "2016-11-23T19:31:00.000Z",
  "landed": "2016-11-23T13:27:00.000Z",
  "airborne": "2016-11-23T19:39:00.000Z",
  "pax": "128",
  "via": [
    "ABR",
    "ACL"
  ],
  "codeshare": [
    "LH123",
    "TG123",
    "AF123"
  ]
}

What is good about this entry level model ? Not too much other than highlighting the benefit to have all info in one document.

Lets look at the problems, at least the highlights. There is quite a number of attributes missing (eg. registration) but here the main flaws:

  • There is no clear concept of the flight as entity. Is it a segment or a complete journey ?
  • No naming convention, more or less random abbreviation for eg. timings.
  • No proper key identifier.No separation of airline code, flight number and suffix, missing schedule departure date (as key).
  • Resource should be an array of objects. Multiple resources with different timings might be in use.
    Same applies to any pax or cargo/load data.
  • Representing VIA and CS information like this might be good enough for a FIDS system but for a mature model we need to break down the whole entity into segments.
  • No links which provide dependencies to other segments, codeshares, arrival or departures.
  • Milestones (timings) should be an array too.
  • No audit information. (Might not in the scope of our model though.)
  • No unique (technical) identifier beyond flight keys.

We will elaborate and finetune in upcoming posts. Stay tuned.

Disclaimer: This discussion, datamodel and application is for study purpose solely. It does not reflect or replicate any existing commercial product.

Image: Creative Commons, DeGolyer Library, Southern Methodist University on The Commons, “DC-3 Aircraft at Houston Municipal Airport, Eastern Airlines”

Advertisements

METAR and TAF Webservice

ozedf

I am regularly exploring the Web Service space in the aviation domain looking for new datasets and feeds. I just stumbled upon CheckWX.com collecting weather messages from various sources and redistribute them. Thanks to Henry who is operating the page (it is not run by a company) we have access to big pool of weather messages via an API. The WS use is free, as long you wont hit the service with more than 2000 calls a day.

METAR (Meteorological Terminal Aviation Routine Weather Report) is a weather forecast in a standard message format, usually used by pilots for the pre-flight briefing. The standard defined by ICAO. The data is collected by weather observation stations.
TAF (Terminal aerodrome forecast), similar to METAR focus on aerodromes and provides a forecast for the next ~24 hours, usually every 6 hours.

You find more information in the ICAO reference “Manual of Aeronautical Meteorological Practice” or in the Airforce “Aircrew Quick Reference to the METAR and TAF Codes”.

Lets build a 5min Android app, similar to the BA Webservice client in a previous post. Thanks to the OkHttp library we get data nicely JSON formatted with a few lines of code. We just need to pass the airportcode (ICAO) to get back the current METAR info (plus some extra info about the airport).

private void callWXweatherService(String airportCode) {

	OkHttpClient httpClient = new OkHttpClient();
	String callURL = "https://api.checkwx.com/metar/" + airportCode + "/decoded";

	Request request = new Request.Builder().url(callURL)
			.addHeader("Content-Type", "application/json")
			.addHeader("X-API-KEY", "{your own API key}")
			.build();

	httpClient.newCall(request).enqueue(new Callback() {
		@Override
		public void onFailure(Call call, IOException e) {
			System.out.println("WS Call failed (1):" + e.getMessage());
		}

		@Override
		public void onResponse(Call call, Response response) {
			ResponseBody responseBody = response.body();
			if (!response.isSuccessful()) {
				final String errRep = "WS Call failed (2):" + response.toString();
			} else {
				try {
					System.out.println("Response " + response.toString());
					String str = new String(responseBody.bytes());
					final JSONObject svcresponse = new JSONObject(str);
					int spacesToIndentEachLevel = 2;
					final String prettyPrintString = svcresponse.toString(spacesToIndentEachLevel);

				} catch (Exception e) {
					e.printStackTrace();
				}
			}
		}
	});
}

The API documentation: api.checkwx.com

screenshotcheckwx

If I have time I will create a better readable gui and integrate with Google Maps.

Disclaimer: This discussion, datamodel and sourcecode or application is for study purpose solely. It does not reflect or replicate any existing commercial product. It is also not fit for or meant for production.

Please note, weather data from CheckWX is not a substitute for an official weather briefing.

Asymetric Encryption 5min walk-through

2018-09-14 13_05_12-343286-PAMSX1-119.jpg - Windows Photo Viewer

Cryptography is one of the key-elements in the Blockchain world. We were already looking into the one-way encryption with hash function in the previous post. Encryption of data to be decrypted again, is equally essential and covered by symmetric and asymmetric encryption. I would not attempt to explain the mathematical  and rather complex background of cryptography, but show with some sourcecode how easy it is to integrate such powerful tools in any solution.

Essential for the asymmetric encryption is the creation of a pair of keys, a public and a private key. The public key can be shared with anyone, it is “public”, it can be used by anyone to create an encrypted message (or data) and send it to a receiver which is the only one holding the private key. This private key is the only way to decrypt the message. This provides a very secure communication mechanism. On the contrary there is Symmetric encryption which uses the same key for encryption and decryption.

Anyone who want to deep-dive into cryptography and algorithm I can refer to books and courses. Otherwise we do an express walk-through here in Java, with the shortest possible sample for key generation (5 lines of code) and encryption/decryption (8 lines of code) of text.

Using the Java core packages java.security it is boiling down to a few methods to create a key pair. You have to choose the algorithm (RSA,DSA,DiffieHellman) and initialize with the keysize. It is obviously a crucial design decision where and how to create the keys and distribute it to the right party.

import java.security.Key;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.PrivateKey;
import java.security.PublicKey;
...

private KeyPairGenerator keyGen;
private KeyPair pair;
static PrivateKey privateKey;
static PublicKey publicKey;
...

keyGen = KeyPairGenerator.getInstance("RSA");
keyGen.initialize(512);
pair = keyGen.generateKeyPair();
privateKey = pair.getPrivate();
publicKey = pair.getPublic();

To encode and decode a text is similar simple. Take note of the binary format, we have to use Base64 to convert to String and back to binary.

	private String encryptText(String messageText, Key key) {

		Base64.Encoder encoder = Base64.getEncoder();
		try {
			cipher = Cipher.getInstance("RSA");
			cipher.init(Cipher.ENCRYPT_MODE, key);
			return encoder.encodeToString(cipher.doFinal(messageText.getBytes("UTF-8")));

		} catch (Exception e) {
			e.printStackTrace();
			return "nil";
		}
	}

	private String decryptText(String cipherText, Key key) {

		Base64.Decoder decoder = Base64.getDecoder();
		try {
			cipher = Cipher.getInstance("RSA");
			cipher.init(Cipher.DECRYPT_MODE, key);
			return new String(cipher.doFinal(decoder.decode(cipherText)), "UTF-8");

		} catch (Exception e) {
			System.out.println(">>>>>> " + e.getMessage());
			return "nil";
		}
	}

Lets run the sample application, create keys, encode and decode a text.
Take note, attempting to decode with the same key fails and the encoding/decoding works with both keys, you can encode with the public or private key and decode with the respective counterpart. Similar to hash we can see the change of 1 char (“My to MY”) changes the cipher completely.

---- KEY GENERATION ----
Algorithm: RSA
Format: PKCS#8
Private Key:
MIIBVQIB(...obfuscated....)Pk/Vp+iTx64pMSQRiXq7UbXX
Public Key:
MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALsDbKCfo55sevpoRL/GjxtlDBFdFLGedfaemCWzh7b/K5ybZbdDrjTVhoyionyGVZuClkZnN5aSRdR6nVXeVL8CAwEAAQ==
------------------------
---- ENCRYPT WITH PUBLIC KEY----
Plain Text: My 2018 Cryptography Sample !
Cipher Text: BkwSG5zczP//7N7AgiFdAyxTvJryPeCLye9zRD1dnKExKR5nABGG3kkiogl8ryujJ+MQB1ISh1EUKSDUm6KYuw==
----------------------------------------------------------
Plain Text: MY 2018 Cryptography Sample !
Cipher Text: Zs8LHPlShaNX/ch8LfrWZrguBJT+WMahMyPQcg06+Nk98i5yd2qyEvP3LfA1QxCmPdFQEPoS7nQxvnPVxE7s0g==
----------------------------------------------------------
---- DECRYPT WITH PRIVATE KEY----
Cipher Text: Zs8LHPlShaNX/ch8LfrWZrguBJT+WMahMyPQcg06+Nk98i5yd2qyEvP3LfA1QxCmPdFQEPoS7nQxvnPVxE7s0g==
Plain Text: MY 2018 Cryptography Sample !
------------------------
---- DECRYPT WITH PUBLIC KEY----
>>>>>> Decryption error
Cipher Text: Zs8LHPlShaNX/ch8LfrWZrguBJT+WMahMyPQcg06+Nk98i5yd2qyEvP3LfA1QxCmPdFQEPoS7nQxvnPVxE7s0g==
Plain Text: nil
----------------------------------------------------------
---- ENCRYPT WITH PRIVATE KEY----
Plain Text: My 2018 Cryptography Sample !
Cipher Text: sjm/MwtAy/MwThhRVKa8lJ0ro1PTHXSK5QfRNLHnYb7X/ezSCLTRPCU7z5TZg7S5ptR2Tvkj4P1J/fk8CNY6xA==
----------------------------------------------------------
---- DECRYPT WITH PUBLIC KEY----
Cipher Text: sjm/MwtAy/MwThhRVKa8lJ0ro1PTHXSK5QfRNLHnYb7X/ezSCLTRPCU7z5TZg7S5ptR2Tvkj4P1J/fk8CNY6xA==
Plain Text: My 2018 Cryptography Sample !

The complete sample sourcecode here on github.
Stay tuned.

Disclaimer: This discussion, datamodel and sourcecode or application is for study purpose solely. It does not reflect or replicate any existing commercial product. It is also not fit for or meant for production.

Blockchain – Big Topic broken down to pieces Part 2 (Bag Custody Sample)

bccloud3

I will continue the exploration of the basic components of the blockchain technology started in the previous post, where I implemented a basic java class to visualize hashing and the linking of blocks (chain). To provide a sample for the aviation industry walking along the use-case of tracking the baggage custody changes during a flight journey .

3. Validation

A blockchain need to be validated otherwise we would not know if data was changed or corrupted. We have to iterate through the sequence of blocks, for this we create an array of bag transactions to allow easy iteration and compare the stored hash of each block with the recalculated hash. If the values match the blockchain is valid, if not it is corrupted (and all subsequent blocks). Ideally this would be event-driven, any change to the blockchain triggers the revalidation. In a real-world implementation changes through the endpoints would not be possible, something we will look at later when working with Corda or Ethereum.

		String currentBagBlockHash = "0";

		String myBagTag = randomBagTagID();
		String myPNR = randomPNR();

		ArrayList allBagTransactions = new ArrayList();
		BagTransaction tempBagTransaction = null;

		// Print Bag Tag (Genesis Block)
		tempBagTransaction = new BagTransaction(myBagTag, myPNR, Entity.NIL.name(), Entity.PAX.name(), 0, "0");
		currentBagBlockHash = tempBagTransaction.getHash();
		allBagTransactions.add(tempBagTransaction);

...

public void checkBlockchainIntegrity(ArrayList allBagTransactions) {
	// Check Blockchain. Compare recalculated hash with hash attribute stored

	for (BagTransaction b : allBagTransactions){
		System.out.print("Block " + b.getBlockID() + " Stored Hash: " + b.getHash() + " -- Calculated Hash:" + b.createHash());
		if (b.getHash().equals(b.createHash()))
			System.out.println(" -- OK");
		else
			System.out.println(" -- FAIL. Blockchain broken.");
	}
	System.out.println(" -- ");
}
	}

For better reading we convert each block into a JSON object.

{
  "timeStamp": "2018-08-26T09:18:09.227Z",
  "blockID": 0,
  "blockHash": "ce41abbf21a162152e30fc511eaf594db43c84948691d12ef78fad6f31fa6043",
  "previousBlockHash": "0",
  "bagTag": "5347241966",
  "pnr": "EONT9T",
  "custodyTransfer": [
    {"transferFrom": "NIL"},
    {"transferTo": "PAX"}
  ]
}
{
  "timeStamp": "2018-08-26T09:18:09.287Z",
  "blockID": 1,
  "blockHash": "6b6073122ef996acc1e3c3e74c3963b5903112800781b4ba88e96baf0a8e2e04",
  "previousBlockHash": "ce41abbf21a162152e30fc511eaf594db43c84948691d12ef78fad6f31fa6043",
  "bagTag": "5347241966",
  "pnr": "EONT9T",
  "custodyTransfer": [
    {"transferFrom": "PAX"},
    {"transferTo": "AIRP"}
  ]
}

Running the validation for a clean and corrupted blockchain

Block 0 Stored Hash: ce41abbf21a162152e30fc511eaf594db43c84948691d12ef78fad6f31fa6043 -- Calculated Hash:ce41abbf21a162152e30fc511eaf594db43c84948691d12ef78fad6f31fa6043 -- OK
Block 1 Stored Hash: 6b6073122ef996acc1e3c3e74c3963b5903112800781b4ba88e96baf0a8e2e04 -- Calculated Hash:6b6073122ef996acc1e3c3e74c3963b5903112800781b4ba88e96baf0a8e2e04 -- OK
Block 2 Stored Hash: a33bbb5ebdec589e9f1b7c7952d6fef3fae2235b5a0600b8755b07040bba8227 -- Calculated Hash:a33bbb5ebdec589e9f1b7c7952d6fef3fae2235b5a0600b8755b07040bba8227 -- OK
Block 2 Stored Hash: 464319fed3a2c49df9a1fbeedae3f3e7c6692db0f6ab97a4c983f9fdf4269e6d -- Calculated Hash:464319fed3a2c49df9a1fbeedae3f3e7c6692db0f6ab97a4c983f9fdf4269e6d -- OK
-- Now corrupt Block 1 by changing the data
Block 0 Stored Hash: ce41abbf21a162152e30fc511eaf594db43c84948691d12ef78fad6f31fa6043 -- Calculated Hash:ce41abbf21a162152e30fc511eaf594db43c84948691d12ef78fad6f31fa6043 -- OK
Block 1 Stored Hash: 6b6073122ef996acc1e3c3e74c3963b5903112800781b4ba88e96baf0a8e2e04 -- Calculated Hash:09fd4223571f079258bd69d935f99cb552cd3ac0854410b81a2313d215929ebd -- FAIL. Blockchain broken.

 

4. Mining Blocks

Now we get to the first more complex concepts of the blockchain, the mining process. We are still operating at a very basic level though with a single node, but we can introduce the mining operation. Without going into too much detail, mining blocks is the step to close/hash a block and creating a new one as part of the consensus process. The incentive to the miner community is a transaction fee given to that miner that solves a hard cryptographic problem first. The more computing power you invested in, the higher chance you have as miner to solve the problem and get the fee. Based on scarcity this consensus approach is called Proof-of-Work (recommended reading). Unfortunately this lead to the current hardware race consuming vast amount of energy for literally no purpose (you let CPU/GPU’s guess numbers basically). This is seen as limitation, together with the long transaction times, and some blockchain start to move to other concepts, such as Proof-of-Stake.

Breaking down the Proof-of-Work to a simple algorithm, we build a hash function that need to create a certain pattern before being accepted. The hash of (data current block + hash previous block + a nonce value) need to have a number of leading “0” in front. The number of “0” is the difficulty and the nonce is an integer value that is changed/increased until the hash matches the required pattern.

public String mineHash(int difficulty) {
	String returnHash = "";
	String tempHash = "";

	String target = new String(new char[difficulty]).replace('\0', '0');
	tempHash = createHash();

	while (!tempHash.substring(0, difficulty).equals(target)) {
		nonce++;
		tempHash = createHash();
	}

	return tempHash;
}

With a growing difficulty (more leading “0”) it takes obviously longer to crack the challenge and with more CPU power you can run through the guessing cycle faster. Some samples below with increasing difficulty running on an ordinary notebook i7 CPU (java executing on a single core/thread in this case).

Difficulty: 1
Attempts: 11
Milliseconds: 10

Difficulty: 2
Attempts: 393
Milliseconds: 30

Difficulty: 3
Attempts: 1.794
Milliseconds: 40

Difficulty: 4
Attempts: 115.756
Milliseconds: 230

Difficulty: 5
Attempts: 3.366.041
Milliseconds: 3.210

Difficulty: 6
Attempts: 5.322.279
Milliseconds: 4.530

Difficulty: 7
Attempts: 76.339.743
Milliseconds: 60.850

Block
{
  "timeStamp": "2018-08-26T10:38:36.734Z",
  "blockID": 0,
  "blockHash": "0000000158d606c953a5df346d459aad949e9dbb4abe74f30ecf225c23112b14",
  "previousBlockHash": "0",
  "bagTag": "7821666095",
  "pnr": "ZHY4RT",
  "custodyTransfer": [
    {"transferFrom": "NIL"},
    {"transferTo": "PAX"}
  ]
}

Have a look at the below website for the current real life difficulty for bitcoin (hashing algorithm not implemented in the simple way we did it here for illustration purpose). Looking at the current difficulty (6,727,225,469,722) you can guess what kind of hardware setup you need to be fast. I gave up beyond difficulty 7 with the algorithm above.

Bitcoin Difficulty

Stay tuned for more blockchain.

Disclaimer: This discussion, datamodel and sourcecode or application is for study purpose solely. It does not reflect or replicate any existing commercial product.

Sourcecode at github

Blockchain – Big Topic broken down to pieces

bccloud2

Blockchain is certainly not the latest buzzword any longer, it moved well forward on the Gartner Hype Cycle, passed the peak of inflated expectations and I am sure we will find it in the trough of disillusionment in the soon to be updated 2018 version. It is picked up by various industries looking for use-cases and applications. Unfortunately we are looking at Blockchain fatigue already, as there is much hype but little visible implementations outside the cryptocurrency space. I prefer projects that implement blockchain as the right tool for a particular problem over the “let’s see which business case we can throw blockchain at” approach.

In the aviation (airport) space I believe Blockchain has its appliance, but as previously stated I wont attempt to build an AODB with Blockchain as “database” for milestones just for the sake of integrating this technology. For some scenarios you certainly need immutability of data, but we can implement this with the means of other immutable data storage. It is also no point implementing a blockchain into a corporate network infrastructure with few nodes under the control of one entity, this does not fulfill the promise of distributed ledger and trust. A few use-cases that I see, usually involving multiple business parties:

  • Baggage tracking from end-to-end (goes well with IATA 753 effective since June 1st)
  • Service and contract management, billing (eg. groundhandler-airline)
  • Aircraft spare parts management (here the track of provenance have a huge impact)

Trying to understand blockchain can be overwhelming, ranging from Satoshi Nakamoto’s original whitepaper to a endless number of books, talks, websites.
One approach to understand the technology is to break it down into smaller pieces that are implementing proven existing technology or algorithms and understand how they come together eventually forming the much more complex blockchain.

I wont attempt to explain blockchain here, this is redundant, plenty of knowledgeable people have written books and articles you can refer to, but split it into some basic easy digestible portions, some coding included. Before attempting to code against real blockchain implementations, like Ethereum or Hyperledger, I will implement the most basic and simple blockchain first.

1. Hash

The most essential element of blockchain is a hash, a digital signature. A hash is a one way encryption,  once something is hashed there is no way to reverse the process and reveal the original text (decrypt it). Using the SHA (Secure Hash Algorithm), the most popular algorithm with its variants of 256, 384 and 512 bit, defined by NIST, we can convert a text (data) of any length to a 256 bit representation (for SHA256) which is represented by a 64 byte hex string. There is a number of libraries implementing the algorithm, below is the Apache DigestUtils version.

	private void testSHA(){

		String sha256hex = DigestUtils.sha256Hex("Jim Smith");
		System.out.println("\n1 SHA256: " + sha256hex);

		String sha256hex2 = DigestUtils.sha256Hex("jim smith");
		System.out.println("\n2 SHA256: " + sha256hex2);

		String sha256hex3 = DigestUtils.sha256Hex("jim smith and the lazy brown fox");
		System.out.println("\n3 SHA256: " + sha256hex3);

		String sha384hex = DigestUtils.sha384Hex("Jim Smith");
		System.out.println("\n4 SHA384: " + sha384hex);

		String sha512hex = DigestUtils.sha512Hex("Jim Smith");
		System.out.println("\n5 SHA512: " + sha512hex);
	}

resulting in

1 SHA256: 65742910cc03889474f1ee2c8f321a105603d0ae2f91070ffd95b35f8da88261
2 SHA256: bfae13266154ec3c4de5c09cf14358305e44f48d2156953723ebbb184a724499
3 SHA256: e5a4a1b8bd88eb7cf8bff9ee5dd235f87ef996262d4d0213c1387f6141ab9574
4 SHA384: c6e76ad773905c1eedb6a0bd9c0b1602a56928d1ce95d70190cd908797466b948dd342aa69dd0343251afece2e48bfc2
5 SHA512: f813c3d9deb66d4999f6839acc60eb6e2fff6a84266c02e0d4b183f5e56d9674c70b0b136f9e1388673cefbc9278f583e3a4c9803ef0c49f9af28aca60dae5ac

Important to notice:
– Change of one character in the original text produces a complete new hash.
– Independent from the length of the original text the hash has the same length.

2. Chained blocks

As the wording implies, there are blocks of information that are linked together. Sounds like a linked list, where every list entry is pointing to the next information. The chained blocks are linked differently, every block points to the hashed previous block.

For illustration I choose a typical baggage journey (simplified).

blockchain

A bag passes different key touchpoints and changes its custody a few times between the various parties during the handling. Every time there is a new milestone event we record it, eg. bag scanned by groundhandler at the chute at what time, and include the hash of the previous milestone. This way the lifecycle from bag drop at departure to bag delivery at destination is recorded in an immutable way and cannot be changed afterwards.

Pitfall: The bag journey is recorded in an immutable way, but the blockchain cannot verify or confirm the milestone actually happened. This falls into the responsibility of the overall design and service orchestration.

Let’s build a very simple application implementing the above blockchain for baggage handling.

A java class BagTransaction representing the bag attributes inclusive timestamp and the custody transfer.

package blockchaindemo;

import java.time.Instant;

import org.apache.commons.codec.digest.DigestUtils;

public class BagTransaction {

	private String bagTag;
	private String timeStamp;
	private String pnr;
	private String transferFrom;
	private String transferTo;

	private long blockID;
	private String blockHash;
	private String previousBlockHash;

	public BagTransaction(String bagTag, String pnr, String transferFrom, String transferTo, long blockID,
			String previousBlockHash) {
		super();
		this.bagTag = bagTag;
		this.timeStamp = Instant.now().toString();
		this.pnr = pnr;
		this.transferFrom = transferFrom;
		this.transferTo = transferTo;
		this.blockID = blockID;
		this.previousBlockHash = previousBlockHash;

		this.blockHash = createCurrentHash();
	}

	public String getHash() {
		return this.blockHash;
	}

	@Override
	public String toString() {
		return "BagTransaction [bagTag=" + bagTag + ", timeStamp=" + timeStamp + ", pnr=" + pnr + ", transferFrom="
				+ transferFrom + ", transferTo=" + transferTo + ", blockID=" + blockID + ", blockHash=" + blockHash
				+ ", previousBlockHash=" + previousBlockHash + "]";
	}

	private String createCurrentHash() {
		String returnHash = "";

		returnHash = DigestUtils.sha256Hex(
				this.bagTag + this.timeStamp + this.pnr + this.transferFrom + this.transferTo + this.previousBlockHash);

		return returnHash;
	}

}

Take note of the hashing method that includes all fields inclusive of the previous hash.

A java class BagDemoApp using the transaction class.

package blockchaindemo;

import java.util.Random;

public class BagDemoApp {

	public static void main(String[] args) {

		BagDemoApp demoApp = new BagDemoApp();
		demoApp.demo1();

	}

	public void demo1() {

		String myBagTag = randomBagTagID();
		String myPNR = randomPNR();

		String currentBagBlockHash = "";

		// Print Bag Tag (Genesis Block)
		BagTransaction bagTransaction1 = new BagTransaction(myBagTag, myPNR, Entity.NIL.name(), Entity.PAX.name(), 1,
				"0");
		currentBagBlockHash = bagTransaction1.getHash();

		// Bag Drop
		BagTransaction bagTransaction2 = new BagTransaction(myBagTag, myPNR, Entity.PAX.name(), Entity.AIRP.name(), 2,
				currentBagBlockHash);
		currentBagBlockHash = bagTransaction2.getHash();

		// Bag SEC Scan
		BagTransaction bagTransaction3 = new BagTransaction(myBagTag, myPNR, Entity.AIRP.name(), Entity.SEC.name(), 3,
				currentBagBlockHash);
		currentBagBlockHash = bagTransaction3.getHash();

		// Display Transactions
		System.out.println(bagTransaction1);
		System.out.println(bagTransaction2);
		System.out.println(bagTransaction3);

	}

	// HELPER METHODS --------------------------------------------

	public String randomPNR() {
		final String alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
		final int N = alphabet.length();

		Random r = new Random();
		StringBuffer tempPNR = new StringBuffer();

		for (int i = 0; i < 6; i++) {
			char nxtChar = alphabet.charAt(r.nextInt(N));
			while ((i == 0) && (Character.isDigit(nxtChar)))
				nxtChar = alphabet.charAt(r.nextInt(N));
			tempPNR.append(nxtChar);
		}
		return tempPNR.toString();
	}

	public String randomBagTagID() {
		String tempBagTag = "";

		long range = 9999999999L;
		Random r = new Random();
		long number = (long) (r.nextDouble() * range);

		tempBagTag = String.format("%010d", number);
		return tempBagTag;
	}

	public enum Entity {

		PAX {
			@Override
			public String toString() {
				return "Passenger";
			}
		},
		GH {
			@Override
			public String toString() {
				return "Groundhandler";
			}
		},
		AIRL {
			@Override
			public String toString() {
				return "Airline";
			}
		},
		AIRP {
			@Override
			public String toString() {
				return "Airport";
			}
		},
		SEC {
			@Override
			public String toString() {
				return "Security";
			}
		},
		NIL {
			@Override
			public String toString() {
				return "nil";
			}
		}

	}

}

Executing the application

BagTransaction [bagTag=1691462171, timeStamp=2018-08-12T08:02:25.745Z, pnr=ICSEAH, transferFrom=NIL, transferTo=PAX, blockID=1, blockHash=3ff736f7158d224db6e2e8ba25f3d50321903cd911646576f442a60f8c5872ed, previousBlockHash=0]
BagTransaction [bagTag=1691462171, timeStamp=2018-08-12T08:02:25.808Z, pnr=ICSEAH, transferFrom=PAX, transferTo=AIRP, blockID=2, blockHash=88dd4a2be3bc90ebce71635bedd6bcb63b326044e4bd49634a859a86458de243, previousBlockHash=3ff736f7158d224db6e2e8ba25f3d50321903cd911646576f442a60f8c5872ed]
BagTransaction [bagTag=1691462171, timeStamp=2018-08-12T08:02:25.808Z, pnr=ICSEAH, transferFrom=AIRP, transferTo=SEC, blockID=3, blockHash=ea7767ddb2dd7c2bfed4d3a038b9249e43df15a0396ff71da717783db9fee3c4, previousBlockHash=88dd4a2be3bc90ebce71635bedd6bcb63b326044e4bd49634a859a86458de243]

Please note, this is the most simple implementation of a blockchain for illustration purpose, it still misses a lot of features to pass to production, eg. mining, proof-of-work, etc.

In a second part might spin this a bit further. Stay tuned.

 

Disclaimer: This discussion, datamodel and sourcecode or application is for study purpose solely. It does not reflect or replicate any existing commercial product.

Design an AODB for a mobile server platform ?

5997469039_0736c09bd7_b

Mobile clients for AODB (Airport Operational Database) products running on tablets and mobile phones are a standard offering today. These clients allow convenient access to operational data while being on the floor, at the tarmac etc. I remember my first mobile AODB project more than 10 years back operating on a proprietary Nokia platform. Today it is not a major technology challenge any longer with current mobile hardware, platforms and libraries available at hand.
The remaining questions are often: How to design the app for a small screen landscape available, but facing a big amount of operational data ? What features does a mobile user really need, what info is key to be displayed, what updates need to be entered while being mobile ? How to simplify user interaction ? How to handle the data synchronization when being offline, aka in areas without wireless or mobile coverage ? Build a hybrid app or go native ?

I believe we at an interim stage of mobile computing at this moment, while current mobile apps replicate desktop applications (screen, mouse, keyboard) with the means of mobile interaction (touch, swipe, pinch,..), we still sorting out the next evolutionary step. Mobile computing on mobile phones, handhelds, wearable computing, smart watches, AR glasses etc. etc. Ruggedized devices became standard while smart watches are still not taking off.
The next (or this !) generation of mobile apps should make use of the mobile characteristics and sensors built-into most mobile hardware. A few ideas: Location awareness, the app should know where the user is at the apron and already open the relevant flight and information or support manual milestone recording. Depending on the users role, tasks should be highlighted to him/her, eg. a service manager that is closest to a problem location should be notified. NFC reader, barcode scanner and AR glasses should assist the user to identify cargo, baggage, vehicles, etc.

Putting aside this considerations, I like to engage in a little thought-experiment:
Can we run an AODB, aka the server, on a mobile phone (Android) ? (without using any external core service, eg. data storage or rule engine etc. Standalone only, allowing other clients to connect, simple interfaces only)
Lets play with pro and con argument and check on the feasibility.

Hardware

This topic triggered the discussion when comparing current hardware in mobile phones with server hardware 20+ years ago when the first generation of AODB products appeared. A standard server of that time was something like a Sun Ultra II with dual 200 MHz processors, 256 to 512 MB of RAM and maybe a SCSI rack with 3x 10GB diskspace, running Solaris, 32bit. Easily priced at 20.000 to 50.000U$ depending on configuration plus various commercial licenses.
Lets look at a current mobile phone like the Huawei P20, Octa-core (4×2.4 GHz Cortex-A73 & 4×1.8 GHz Cortex-A53) with 4GB of memory and 128GB of SD card space, coming at 600U$.

Operating System

Android is obviously not a server OS, it does not give us control over settings that we can rely on under Linux. We cant assign memory and priority, it is actually the OS that controls apps and services, going to the extend of terminating unused apps and similar. The only way of having a persistent app running is as service. Apps are living in sealed sandboxes, only a rooted device would give us more control.

Solution architecture

We cant build a multi-tier solution with the classical database, business logic and frontend layers. Android enforces monolithic applications, the only way to escape this is by building services and relying on ICP (AIDL, Intents, Binder). Anonymous shared memory is only available in Android 8+.

Database and Application/Web-Server

App-server ? Easy to answer, it does not exist. Maybe simple http server is possible.
Database ? Only a few solutions at hand, either the built-in SQLite, not really a DB known for performance, or some alternatives, mostly key-pair and relational DB’s and some NoSQL DB (Comparison chart here).

Scalability

We cant scale vertical, no adding of CPU’s or memory possible. Horizontal scaling would not be easy, unless we deploy more mobile phones to outsource certain services, but implementing a load-balancer would not be possible.

Availability

In terms of network connection we are limited to 1 wireless or (!) 1 mobile connection at a time, no redundancy. Power-failure is less of a problem, we have built-in battery that could last at least 1 hour under heavy usage. Android as OS is quite stable, it can run for prolonged periods, though it is uncertain if services running permanently with load create a problem.

Integration

Integration of interfaces with other systems is a bit more challenging. Though it is no problem to consume webservices, download from ftp server or receive emails as part of interface client, we will have a hard time to provide an interface, eg. to offer a webservice. There is no ESB running on Android.

Conclusion

It is definitely possible to run a very lightweight AODB solution without lots of fancy bells and whistles on a mobile phone, ideally to act both as server and client integrated into one solution (app). All under the premise to limit our requirements to a basic set of features like managing schedule, daily operations, milestone handling, simple resource management.
The longer I review this idea the more arguments I collect against this use-case, the platform is too limited to allow scaling, does not provide real server features and will not be able to run heavy services like rule engines, ESB, etc.
Maybe feasible for small scale operations at an airport with few commercial flights a day, some GA, few users and utilizing third party services in the cloud for billing, ESB, etc.

I suggest we rather invest our thinktank energy in building a serverless AODB by using orchestrated microservices and use the mobile platform solely as client.

Image: Creative Commons, National Library of Ireland on The Commons, “St. Albert at Dublin Airport, circa 1950”

Aviation API – Airline and Airport Webservices

The term API (Application programming interface) is not new, basically a defined access to a set of methods, subroutines and data-types made available by one component/service to be used/consumed by another component or application. You can think of API as the car manual and the library being the engine under the hood.

Since the early days of tinkering with the Windows DLL hell more than 20 years have passed, we have better tools and standards by now. In the space of native apps for Windows, Android and iOS we still work with libraries and SDK’s (still can be challenging when resolving dependencies and deployment).
In the world of web applications today we look mostly at RESTful Webservices responding in JSON or XML, a rather straight forward implementation, the only complexity depends on the authorization mode or the mapping of attributes. A lot of websites, portals, services or products expose their WS for usage by third parties, from Salesforce, Ebay, Amazon to Twitter and infinite more. While these mentioned samples operate independently or work as standalone services there is not much need for standardization of the payload, aka the structure of attributes, naming conventions etc. For others business domain there is a need of standardization of such, despite the availability of AIDX, AIDM and a few more data exchange models, there is no standard widely used in the (public) WS space in aviation.
I have to highlight in ACI ACRIS a Semantic Model is being actively developed and the Open API Shop exists as project as well.

I researched what webservices are currently offered to the public by airports and airlines, excluding the API’s of system vendors and travel platforms.
So far I found these webservices (last update 2018-07-06):

Airline

International Airline Group
British Airways, Iberia, Avios
https://developer.iairgroup.com/
Lufthansa https://developer.lufthansa.com/
Airfrance, KLM https://developer.airfranceklm.com/
Alaska Air
https://developer.alaskaair.com/
Transavia https://developer.transavia.com/
FlyDubai https://developer.flydubai.com/
Virgin Australia
https://developer.virginaustralia.com/
Ryanair https://developer.ryanair.com/
Turkish Airlines
https://developer.turkishairlines.com

Airports

Schiphol Airport
https://developer.schiphol.nl/
San Francisco Airport
https://www.flysfo.com/api
Frankfurt Airport
https://developer.fraport.de/
Svedavia Airports
https://apideveloper.swedavia.se/

Others

FAA US
https://app.swaggerhub.com/apis/FAA/ASWS/1.1.0
Flightaware
http://flightaware.com/commercial/flightxml

The usage terms and price models vary but basically all give some kind of developer access to evaluate the services and data at no cost.

To compare a few of the services using a flight status related call, omitting the authentication. The lack of standard, putting aside JSON response format and date format, is quite obvious. It might not be relevant in the space of individual apps to have the same response format, but if you want to combine data from various sources you have to handle the formats separately. Even the request format with the query parameter differs.

British Airways

https://api.ba.com/rest-v1/v1/flights;departureLocation=FRA;startTime=06:00;endTime=11:00

ba

Swedavia Airports

https://api.swedavia.se/flightinfo/v2/query?filter=airport eq ‘ARN’ and scheduled eq ‘180713’ and flightType eq ‘A’ and flightId eq ‘DY4572’

IMG122

Lufthansa

https://api.lufthansa.com/v1/operations/flightstatus/LH778/2018-07-13

IMG123

I also tried with RyanAir (still waiting for approval to get api key), Turkish Airlines (no flight status API), Schiphol (no webservice test available on the website).

Image: Creative Commons, Robert Yarnall Richie Photograph Collection, “Models with Oldsmobile Automobile, Lockheed 10B Electra, Delta Air Lines, 1940”