Android

Germany’s Covid-19 Tracing App

Posted by

0

Finally the contact tracing up is here. With big media attention the app was launched to public 2 days back, after the bumpy start with a complete strategic change from centralized to decentralized model. SAP and Deutsche Telekom cooperated and created an app-server environment in less than 2 months, chapeau for that performance. It comes at 20 million Euro, plus 2 to 3 million monthly for Deutsche Telekom, mostly to operate the hotline (not sure if they actual build an office building first, but anyway). Some might claim a start-up could have done it for a fraction, yes, maybe, would it be ready ? Would it be able to provide a system that can talk to the public healthcare system to create TAN’s to push alerts ?

You can download for Google and Apple. Interestingly the app was downloaded more than 1 million time within 2 days, after 3 days supposedly 8 million times and got more than 30.000 ratings/reviews (Android only) in that short timeframe (mostly positive) !
The team seem to work agile, they pushed out an update within 72 hours.

You can review and download the code at Github. I must admit, the documentation is really good, no one can claim this is not transaparent. Despite still so many people claim it would be used for tracking people and their whereabouts.
First thing to do ? Let’s look at the sourcecode and the underlying libraries by Apple/Google. Did not manage to build it with Android Studio 3.4, had to update to 4.0 to get it built and run on the phone, though it would fail after the initial info screens due to the missing Exposure Notification API on the phone. Guess that will be fixed once the Playstore app is updated.

Findings:

  • Permissions
    As planned and communicated only the bare minimum to make this app work. You can see it does not require the coarse location permission usually required by BLE.
  • Libraries
    A few of the standard libraries: ZXing Embedded, Joda Time, Room, SQLCipher, detekt and a some others.
    Most importantly Google’s Exposure Notifications API
  • Why would you disallow the user making screenshots, if everything is transpraent and opensource ?
  • Once you installed the apk file through Android Studio you cant install the official app, even after removing the app installed through adb.

Conclusion: Full transparency. It did materialize quite fast, too late for the first wave (no app can be ready for something totally new) but certainly in place if a second wave or individual hot-spots appear. I still have doubts if we reach a 60% penetration nut the initial figures are looking good. You can still argue about the quality of measuring the signal strength of Bluetooth and how many warnings we will see.

Daily Tech Observations 9

Posted by

0

Tracing Apps Overview

About 35 countries released apps in the COVID-19 context, roughly half of them actually perform some kind of tracing or tracking using either GPS, Bluetooth or both. I reviewed some of them with the information that you find in the Google Play Store in regards of rating, no. of installations and permissions. I could not find any app that has a penetration of more than 5% (Android only), rather lower, and the success of such an app grows with the userbase.

Random samples:
(Numbers for Android apps only, Apple is not releasing the number of installations)

  • Stopp Corona (Austria), ~9.000.000 population and 100.000+ downloads
  • eRouška (Czech Republic), ~11.000.000 population and 100.000+ downloads
  • Aarogya Setu (India), ~1.300.000.000 population and 50.000.000+ downloads

Reference: Population by Worldbank

I noticed some apps are very generous with permissions, also asking for identity, phone number, contact list, all on top of GPS and Bluetooth.

Tracing Apps Details

Thanks to AppBrain you can get more information about individual apps. You can see the underlying libraries used and communication channel. It seems most of the apps make use of Google Firebase Messaging for the push part.

Sample: Dev libs for Stopp Corona (Austria)

Here as sample the details for the Stopp Corona App from Austria taken from the respective AppBrain page. We can see GCM (now Firebase) for messaging, the cryptographic lib Bouncy Castle, the JSON lib MOSHI.
At least none of the apps I have looked at so far, use social or Ad network libraries.

German Tracing App Status

While other countries have a headstart, Germany is falling behind. Despite news stating German Telekom and SAP joining the initiative, there seems to be no schedule, commitment and definitely no budget yet. I fear this will become a humongous project with 90% consultancy. We have existing open source applications following DP-3T as guidance, a small team of less than 15 people can produce an app in short time, and still comply with GDPR and security in place. My take.

COVID-19 Data

Latest Google Community Report was released on May 1st.

Stay safe and tuned.

Image by Free-Photos from Pixabay

Overview Covid-19 Tracing Apps and Protocols

Posted by

2

Last Update: 2020 May 6th

Protocols and Initiatives

Open Source Apps, Code or Demo/POC

Mobile apps or projects with public accessible sourcecode.

Propietary Apps

Android Apps in the Google Playstore.
To notice quite a number of apps receive rather average reviews and quite some negative comments.
Not all apps are tracing apps, but information portals or self-checks.
[Average rating – number of ratings – installs] (As of April 29th)
Trac(k)ing apps are highlighted with * (Bluetooth) and ** (GPS). Specific contact tracing apps highlighted in bold.Other applications are general health, guidance, self-check or crisis apps.
Source: Respective app page in Google Play Store

Document References

Papers, articles, PhD Thesis, Research Documents. Focus on Wifi/Bluetooth/BLE technology.
No specific order, no attempt to be complete, quite a number of documents are discussing these topics.

(Image by Samuele Schirò from Pixabay)

Daily Tech Observations 7

Posted by

0

PEPP-PT

There has been some movement in the Pan-European Privacy-Preserving Proximity Tracing project over the last few days. Earlier it was announced an app would be released after the Easter period. This has not happened so far, some of the project partners have even pulled out (reference). The project is driven by the Arago founder Chris Boos.
Though the app is not out, both documentation and some sourcecode has been released under Mozilla Public License on github. This is a good move, it creates transparency and allows to fork the project if needed. I have not seen other projects in the same tracing space to release their sourcecode. Time to have some hands-on with the sourcecode, more in an upcoming post.

ROBERT Protocol

The ROBust and privacy-presERving proximity Tracing protocol (ROBERT) by Inria and Fraunhofer Institute. Both are member of the PEPP-PT tracing project but the documentation can or should be the base for any implementation of a tracing app. Find the documents at github. This is the only way forward to create transparent apps that will be accepted by general public. Highly recommended reading.
It is key to differentiate between the centralised and decentralised approach, read more here.

DP-3T

As the ROBERT protocol is a solution with a central component, there is also a group proposing the a decentralised approach, DP-3T – Decentralized Privacy-Preserving Proximity Tracing. This proposal is aligned with the mutual Apple-Google proposal. Also highly recommended reading. Sourcecode is available on github as well.
Drop by the comic style explanation by Nicky Case.

By Nicky Case (CC-0)

TCN Protocol

Similar to DP-3T , there is TCN (Temporary Contact Numbers,) a decentralized, privacy-first contact tracing protocol. Find the specification here.

Open-Source Corona Apps

Two mobile apps have been released in the US

COVID-19 Data

More Research and Whitepapers

DIY Project: Create a Tracking and Tracing App Part 3

Posted by

0

In this part we will have a short excursion into the world of radio signals, like wireless or bluetooth signals. As part of the tracing solution we must estimate the distance between two devices, it makes a difference if we stand next to a person (less than 2m) or being 5m and more apart for a potential transmission. The tracing app shall record only other devices in the nearby area, otherwise we will face a tsunami of false warnings after recording everyone in a 10m radius. The only way to measure or anticipate the distance is the signal strength of the received beacon signal.

Bluetooth Peripheral Mode

Bluetooth classic, made for higher speed and permanent connections, uses more energy and requires pairing before exchanging data between the devices (see previous post). We need to use the peripheral mode which was introduced with Android 5 (Lollipop) in 2014. The peripheral mode is mostly used by health devices, pedometers, etc. By today (2020) most Android phones should support this mode which is the key component, known as Bluetooth Low Energy Advertising, for the tracing app.
Bluetooth 5, supported since Android 8.0, introduced significant improvements to the BLE mode (reference). In the next post we will explore the BLE advertising and the related services.

Simple test to check if the Android device supports advertising

private BluetoothAdapter bAdapter = BluetoothAdapter.getDefaultAdapter();
..
if (bAdapter.isMultipleAdvertisementSupported())
	Log.i(TAG, "MultipleAdvertisementSupported supported.");
..

About Signal Strength

Theory

To be more precise we have to look at the strength of the received signal, also called RSSI (Received Signal Strength Indication), the measurement of power in a radio signal, measured in dBm. In short, the receiving device can measure the power of the signal and approximate the distance. Sounds simple, but it is not, radio signals is a huge field in science and research and I won’t attempt to replicate this in a blog post. The RSSI value often ranges between -100 and 0 dBm (in our context here), where -100 is the weakest signal and values near 0 the strongest.
Some links with references below for the interested reader. The main challenge is the signal strength depends on a number of parameters, the sending power, the distance (obvioulsy), external factors like reflection, absorption, interference and diffraction. It is very much an approximation, especially as we are talking about unknown devices (mobile phones) emitting the signals and not defined devices like industry beacons. In the literature you find a formular that estimates the distance in meters:

As you can see we do not have proper values for RSSId0 and Eta/n because we lack of reference devices and reference environments. We will experiment with values in the field test below.

References:

Android

For both Bluetooth and Bluetooth LE we can read the RSSI (values between -127 and 126) easily (Android Developer Documentation). See previous post for complete method.

For Bluetooth Devices

BluetoothDevice device = intent.getParcelableExtra(BluetoothDevice.EXTRA_DEVICE);
int rssi = intent.getShortExtra(BluetoothDevice.EXTRA_RSSI,Short.MIN_VALUE);

For Bluetooth LE devices

public void onScanResult(int callbackType, ScanResult result) {
	System.out.println(result.getRssi());
}

The method to calculate the distance based on above formular

double getDistance2(int rssi, int rssid0, float eta) {
	 return Math.pow(10d, ((double) rssid0 - rssi) / (10 * eta));
}

Field Test

Devices: Huawei P30 and Ubudu Beacon.

I use the app to read the RSSI value at the reference distance 1m.
In the first round I setup it outside at a grass field without surrounding building, walls etc.
The average RSSI value is about -83dBm with values ranging from -104 to -77dBm.
The second round in an office like environment, a room of about 3×3 metres. Now we have an average value of -51dBm with values ranging from -79 to -35dBm. In a second room I get -88dBm, -83dBm

RSSI Values at 1 meter distance

Now going back to our formular we calculate the distance with reference 1m RSSI value of 75dBm (best guess) and an eta of 2 (found this value when researching). Now setup again a 1m distance situation and check the caculated distance.

Calculated distance at 1 meter distance

This run with 2,1m average value differs 100% to the real 1 meter distance, the values having outliers up to 30m without touching the device or moving anything. If we need to rely on these values we need to capture at least 100 signals and average them to get anywhere near the real distance. I doubt changing eta and the reference RSSI will help as the RSSI value comes with these extreme outliers.

A few more random tests a different distances I come to the conclusion (with this specific test setup), the RSSI wont help us to measure the distance between two mobile phones, aka 2 persons properly. At most we can anticipate with an array of measured values and the the average if the device is less than 5m away, aka falls into a potential transmission candidate group.

Test Setup at 50cm resulting in 18cm average distance.

Header Image by Juanma_Martin from Pixabay

DIY Project: Create a Tracking and Tracing App Part 2

Posted by

1

The tracing of contacts through mobile apps became the Number One hot topic in the last few days, governements and institutes of the EU countries are still working on technical solutions to trace transmissions of SARS-CoV-2 (though a bit late for the first wave that has hit most countries worldwide). At the same time there is an intense debate about these apps in terms of data usage, privacy, etc. The apps wont stop the spread or protect the person using the app but they should help to keep the situation under control in the times to come, maybe as a permanent tool to stay for a long period. Even more important not to build a tracking tool following examples of more authoritian states, but to have a solution that protect privacy.

In this blog series, looking at the technical aspects, we still touch both tracing and tracking for the matter of the discussion. In the last post we only touched the Bluetooth basics, now get into discovering nearby devices.

Android to discover Bluetooth devices

About device discovery

  • Discovery of Bluetooth devices is the step before pairing and coummunicating with another device. We can scan for nearby devices without the other devices (its owner) noticing it.
  • But for classic Bluetooth, the device need to be set to discoverable by its user, usually only for a limited period. It is consuming additional energy and would drain the battery faster if left on permanently (putting aside security concerns, see references).
  • BLE works like a beacon permanently being discoverable, certain location type application work like this, eg. to help navigate in buildings equipped with beacons.
  • The 3 key device attributes when discovering devices:
    – Name: Not unique, just a label, can be set/changed by the user.
    – MAC: The unique identifier (see previous post)
    – Signal strength in dBm (more about this later)

Discover classic Bluetooth devices

We need to register a broadcast receiver and listen to the intents for discovery start and end. The discovery need to be triggered, it will run for about 12 seconds.

Register BC Receiver

private void initBCReceiver(){
	final BroadcastReceiver mReceiver = new BroadcastReceiver()
	{
		@Override
		public void onReceive(Context context, Intent intent){
			String action = intent.getAction();
			if (BluetoothDevice.ACTION_FOUND.equals(action))
			{
				BluetoothDevice device = intent.getParcelableExtra(BluetoothDevice.EXTRA_DEVICE);
				int rssi = intent.getShortExtra(BluetoothDevice.EXTRA_RSSI,Short.MIN_VALUE); // dBm
				System.out.println("Found: " + device.getName() + "," + device.getAddress() + "," +  rssi);
			} else if (BluetoothAdapter.ACTION_DISCOVERY_STARTED.equals(action)){
				System.out.println("ACTION_DISCOVERY_STARTED");
			} else if (BluetoothAdapter.ACTION_DISCOVERY_FINISHED.equals(action)){
				System.out.println("ACTION_DISCOVERY_FINISHED");
			}
		}
	};

	IntentFilter filter = new IntentFilter();
	filter.addAction(BluetoothDevice.ACTION_FOUND);
	filter.addAction(BluetoothDevice.ACTION_PAIRING_REQUEST);
	filter.addAction(BluetoothAdapter.ACTION_DISCOVERY_STARTED);
	filter.addAction(BluetoothAdapter.ACTION_DISCOVERY_FINISHED);

	registerReceiver(mReceiver, filter);
}

Now trigger the discovery

bAdapter.startDiscovery();
Discover Classic BT devices

Discover BLE devices

The BLe devices (beacons) constantly send their signal, we can pick it up in an async thread. The Android BT library supports this with less than 15 lines of code to capture the devices. Implement the callback and start/stop the scanning.

private ScanCallback leScanCallback = new ScanCallback() {
	@Override
	public void onScanResult(int callbackType, ScanResult result) {
		System.out.println(result.getDevice().getAddress() + "-" + result.getDevice().getName() + " rssi: " + result.getRssi() + "\n");
	}
};

public void startScanning(View view) {
	System.out.println("start scanning");
	AsyncTask.execute(new Runnable() {
		@Override
		public void run() {
			btScanner.startScan(leScanCallback);
		}
	});
}

public void stopScanning(View view) {
	System.out.println("stopping scanning");
	AsyncTask.execute(new Runnable() {
		@Override
		public void run() {
			btScanner.stopScan(leScanCallback);
		}
	});
}
Discover BLE devices

Interesting observations:
– The MS Designer Mouse is operating in both classic and BLE mode.
– The signal strength of devices can change without being physically being moved.

Conclusion

  • The Bluetooth classic mode is not feasible for the tracing requirement. It would drain batteries quickly and we cant disnguish between phones and other devices using solely the MAC (though we could identify manufacturers).
  • We need to consider the BLE peripheral model for our tracing app. Remember, we need to capture the unique key from another nearby user of the app, we cant achieve this without basic 2 way communication between the two apps.

Fun Facts

Stay safe and tuned..

References

Image by Free-Photos from Pixabay.

DIY Project: Create a Tracking App Part 1

Posted by

1

The discussion about mobile phone location tracking of people and tracing back to potential transmissions is one of the hot topics at moment. In Germany we could expect an app officially being launched towards end of April. I attempt to go through the technical considerations by myself. A hands-on coding excursion with Android to use Bluetooth to scan nearby devices and exchange data with them.

The most basic requirements for a tracking app to be successful:

  • A person need to posses and carry a switched-on mobile (smart) phone.
  • The phone must have GPS and Bluetooth feature and both being enabled.
  • The location need to be recorded as fine-grain as possible. Use of GPS is mandatory, the celldata is way too coarse (see previous post). Though we might consider to skip location completely and rely on the paring of fingerprints solely, depending on the approach.
  • Approach 1: We record the location and time of a device (aka person) and transmit the data to a server immediately and try to match data with other devices on the server. Hard to implement in a GDPR compliant way and users most likely wont buy in.
  • Approach 2: We record the location and time on the device and any digital fingerprint of devices nearby. This anonymous pairings we transmit to the server. Once one device is flagged as infected, the server can flag any other device “paired” previously and push (or pull) a notification to the impacted devices. This way most data remains on the device. A more GDPR compliant way of solving this. Some details need to be worked out though in regards of matching and informing the respective user.
  • Approah 3: Even better if we could rely solely on the fingerprint of nearby devices and the timestamp.
  • The more user we have in the system, the bigger the impact and the chance to trace and inform and potentially stop spreading further.
  • We must have a mean to report an infection and inform affected other users (and still stay within the boundaries of GDPR).

Before walking into the Bluetooth space, some facts:

  • The not-for-profit organisation Bluetooth Special Interest Group (SIG) is responsible for thedevelopment of Bluetooth standards since 1998. (Wikipedia)
  • There is a regular update to the Bluetooth standard, by January this year SIG released version 5.2. It takes time for the hardware manufacturers to adopt the newer standards.
  • We need to distinguish between Bluetooth Classic and Bluetooth Low Energy (BLE). BLE was introduced with version 4 and supported by Android 4.3.
  • Bluetooth Classic is designed for continous short distance two-way data transfer at a speed of up to 5 Mbps (2.1 Mbps with Bluetooth 4). BLE was made to work with other devices at a lower speed and greater distance.
  • Android 8.0 onwards support Bluetooth 5 which is a significant milestone for Bluetooth technology in terms of range, speed and power consumption.
  • It is not possible to programmatically check the supported Bluetooth version in Android, though you can check if BLE is available on the phone.
  • The MAC address of the Bluetooth adapter is fixed and can’t be changed (except for rooted phones). This way it becomes the digital fingerprint.

Are we running out of MAC addresses ?

MAC addresses (used by ethernet, wifi and bluetooth adapters), as per IEEE 802 definition, have 48 bits (6×8 bytes).
Sample AC:07:5F:F8:2F:44
This would result in some 281 trillion (2^48) possible combinations, but the first 3 bytes are reserved to identify the hardware manufacturer. For above sample AC:07:5F it is Huawei. The remaining 3 bytes are used as unique identifier, resulting in only 16 million (2^24) unique devices. Quite likely this number would be used up more or less quickly by a big manufacturer. In reality we also could have 16 million unique manufacturer ID’s, Huawei owns about 600 of these, giving a total of currently 10 billion devices. We need to consider this numbers when we talk about unique fingerprints (MAC), though it is unlikely at a country level to have duplicates. In Germany we have ~83 million citizens and about 142 million mobile phones from different manufacturers, small chance that two persons (actually using the tracking app) will have the same MAC address.
You can check/download the identifiers here.

Lets get started with some coding..

Basic: Android to list paired devices

Before we jump into the more complex discovering, pairing and communication between devices (using threads,) we start with the basics. Lets enumerate the paired devices.

Required Permission

At minimum access to coarse location (since Android 6) is needed since Bluetooth can be used to derive the users location. I skip the code to request the permission, only location access being a critical permission. (complete code will be pusblished at the end).

<uses-permission android:name="android.permission.BLUETOOTH"/>
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN"/>
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />

Check and Activate Bluetooth Adapter

public class MainActivity extends AppCompatActivity {

    private static final String TAG = "bt.MainActivity";
    private BluetoothAdapter bAdapter = BluetoothAdapter.getDefaultAdapter();

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        checkAndRequestPermissions();

        if(bAdapter==null){
            Log.i(TAG,"Bluetooth not supported.");
        } else {
            Log.i(TAG,"Bluetooth supported.");
			
	        if(bAdapter.isEnabled()){
				Log.i(TAG,"Bluetooth enabled.");
				if (!getPackageManager().hasSystemFeature(PackageManager.FEATURE_BLUETOOTH_LE))
					Log.i(TAG, "BLE not supported.");
				else
					Log.i(TAG, "BLE supported.");
			} else {
				Log.i(TAG,"Bluetooth not enabled.");
				startActivityForResult(new Intent(BluetoothAdapter.ACTION_REQUEST_ENABLE),1);
			}
        }
    }
..

List existing pairings

Quite simple to iterate through the existing pairings and list their name and MAC Address

private void showPairedDevices(){
	Set<BluetoothDevice> pairedDevices = bAdapter.getBondedDevices();
	if (pairedDevices.size() > 0) {
		for (BluetoothDevice device : pairedDevices) {
			String deviceName = device.getName();
			String deviceMAC = device.getAddress();
			Log.i(TAG,"Device: " + deviceName + "," + deviceMAC);
		}
	}
}

I/bt.MainActivity: Device: HUAWEI P20,AC:07:5F:XX:XX:XX
I/bt.MainActivity: Device: moto x4,0C:CB:85:XX:XX:XX

In the next post we will discover nearby Bluetooth devices and setting up a communication channel between two devices.

Stay tuned for more tracking..

References

Image by Brian Merrill from Pixabay

Daily Tech Observations

Posted by

0

As the pandemic crisis continues, more discussion, data exchange and research is happening and progressing in the digital space. I wont mention the massive increase of security threads here (reference info at Trendmicro), but rather look at the non-malicious activities.

PEPP-PT

The PEPP-PT (Pan-European Privacy-Preserving Proximity Tracing) project around a number of prominent research institutes across Europe is working on a proximity-based solution utilising BLE technology embedded in mobile phones. It will be in line with GDPR regulations and to be used on a voluntary base. It is supposed to track and report your whereabouts adn nearby other app users to a server anonymously only, and inform you when you have been close to an infected person, all that without using personal information, which is the key concern of many parties. A key element for the success of such a solution is the penetration factor. It need to build up a database with a significant number of users and traces. Instead of releasing yet another app, they try to piggyback into existing apps, such as NINA (an app to publish and warn about local dangerous incidents in Germany). It has not been published yet, I assume the technical field test was successful is reported, still they have to sort out the communication channels in the case of an infected user.

COVID-19 Apps

There are no new apps in the Google Playstore since my last post, though I have to correct the app I mentioned previously, TraceTogether, only appears for Singapore based accounts. In the German Playstore we see two apps, the app “COVID-19” transmits the status of a COVID test to the respective user, only reducing the need to physically visit a place to retrieve the results. The other app, Coronika, tries to assist individuals to trace their locations and contacts.

Google to hand over anonymous location data

Google and the other big players are in active talks in various countries with the respective authorities about releasing data, either aggregated or anonymous or both. Depends very much on the local regulations. In the context of stopping the pandemy this would provide valuable insights. Aggregated data can help to identify streams of persons or hotspots of too many people in the same area or similar. If anonymization alone is good enough to protect personal data, I would question, the trace that everyone leaves with an Android phone (location services enabled) would easily allow to identify an individual or a small group, you just look at regularly visited places to identify someone’s home or office etc.

You know you can not only see your traces in Google Maps but also export the data (as well delete it permanently if you want) with the Take-Out feature?

Your Timeline – Google Maps
Take Out – Personal Google Maps Data

You are looking for some well formatted data to play with ? Download your own location data and have some hands-on datascience exercise. Easy to request and download, all nicely packaged in self-explaining JSON formatted monthly files.

Stay tuned and safe !

DIY Tracking and Tracing

Posted by

0

In the current situation to trace people with an infectious disease is key and quite some manual Sherlock Holmes style work to find the traces of a patient in a certain region and who her/she/it met and potentially infected.

Technology could be at help here. GDPR is protecting personal data and the wherabout’s of a person falls into this category, but GDPR describes the current situation.

Reference (eur-lex.europa.eu)

  • Article 46
    The processing of personal data should also be regarded to be lawful where it is necessary to protect an interest which is essential for the life of the data subject or that of another natural person. Processing of personal data based on the vital interest of another natural person should in principle take place only where the processing cannot be manifestly based on another legal basis. Some types of processing may serve both important grounds of public interest and the vital interests of the data subject as for instance when processing is necessary for humanitarian purposes, including for monitoring epidemics and their spread or in situations of humanitarian emergencies, in particular in situations of natural and man-made disasters.
  • Article 6 (d) – Lawfulness of processing
    processing is necessary in order to protect the vital interests of the data subject or of another natural person;

Some data (geolocation data) has been provided by German Telekom of its mobile customers to the RKI for people movement research, no individual identifiable data though. Please note it is a small subset and anonymized, there are quite a number of social meida posts and comments informing wrongly.

This is the right thing to do in this situation, the data exist and can speed up the containment. Though it need to be ensured that the date is not used for other purposes or beyond the crisis (as long personal data is stored). Unfortunately this could serve as reference for more (meta-)data harvesting by authorities in future without immediate purpose.

Google and Apple would be in the best position to track down individuals and find the cross roads of tracks and potential infection clusters etc. Most of the people have Android or iOS phones and even not all of them have GPS enabled, the devices still log into the cell towers.

In an earlier project I created an app that is recording the cell tower information while you are on the move. (The app is not in the playstore due to GDPR). Using the recorded cell tower info and matching with the opencellid celltower geolocations I created a map of my own movements in Python using the Folium library.

The geolocation data of the individual towers coming from opencellid is not always 100% accurate on the spot but certainly good enough to to estimate the track. Individual points are too coarse, in this sample dataset, created when I was driving along the highway (red line) the phone connected to various celltowers along the way, even to towers further away from my route. Conclusion: Only the complete dataset can help datascientists to estimate my track and potential contact points with other people.

It will be much harder to track down an individual in a urban environment with 100’s of cell towers, see OpenCellid sample for Frankfurt. I would guess that is the reason RKI only tries to identify streams of people between places.

In China or Israel the technology is already used to pin down individuals. I leave it to you to comment. In Europe, in the interest of personal data protection, an innovative approach would be to (continue to) trace (yourself) but allow the individual to be alerted or verify against an open dataset to be informed about clusters etc. Though, as mentioned before, at some stage the authoroities should make use of the data as long it matches laws and is purposeful.

Further Reading: Wired Magazine

Android and Speech Recognition (2)

Posted by

0

In part 2 about speech recognition we do the reverse, instead od openly calling Google Search or integrating the speech recognition intent (prominently showing the Google logo/splash screen) we call the same recognition method in a headless way, making the experience more seamless. To make it menaingful in the aviation context, we will request BA flight information (through IAG Webservices) through audio.

(Please note, the below code snippets are incomplete and only highlighting the key methods, the complete sourcecode you find at Github, see link at the end of the post.)

Calling the speech recognizer

sr = SpeechRecognizer.createSpeechRecognizer(this);
sr.setRecognitionListener(new listener());

Intent intent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
intent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL,RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);

intent.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS,10);
sr.startListening(intent);

Implementation of the speech recognition class

class listener implements RecognitionListener
{
	public void onResults(Bundle results)
	{
		ArrayList data = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
		StringBuffer result = new StringBuffer();
		for (int i = 0; i < data.size(); i++)
		{
			Log.d(TAG, "result " + data.get(i));
			result.append(i + ":" + data.get(i) + "\n");
		}
		textViewResult.setText(result);
	}
}

With this approach we get the usual 2 acoustic sounds signalling the begin of the speech recognition phase and the end (after a short time out).

If we need to create a hands-free user experience, avoiding the user to touch the screen, we will make use of the play or call button that you usually find on headsets. We can capture the event that gets fired when pressing these hardware buttons too.

Capture hardware button events

@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
	Log.v(TAG, event.toString());
	if(keyCode == KeyEvent.KEYCODE_HEADSETHOOK || keyCode==KeyEvent.KEYCODE_MEDIA_PLAY){
		Log.i(TAG, event.toString());
		listenHeadless();
		return true;
	}
	return super.onKeyDown(keyCode, event);
}

Text-To-Speech

The missing link to the hands-in-the-pocket experience is the audio response by the app through our headset. We will add the standard Android Text-To-Speech (TTS) implementation.

textToSpeech = new TextToSpeech(getApplicationContext(),null);
...
textToSpeech.setPitch(1.0f);
textToSpeech.setSpeechRate(0.80f);
int speechStatus = textToSpeech.speak(textToSpeak, TextToSpeech.QUEUE_FLUSH, null, null);

if (speechStatus == TextToSpeech.ERROR) {
	Log.e("TTS", "Error in converting Text to Speech!");
}

A remark about text to speech engines, the default speech engines are not very pleasant to listen to, for an application speaking to users repeatedly or over a long time I recommend to look for commercial TTS engines (eg. Nuance, Cereproc,..). Check out the sound samples at the end of the post. TTS engines are usually produced per language. For a multi-lingual application you need to cater for every language.

Adding Webservice

To make this sample app more business relevant (airport related), we use the spoken text to request for flight data. For this purpose I will reuse the BA webservice call that was used in an earlier post. Do not forget to add permission for internet access to our app.

We straight away hit the first challenge here, we receive a text version of the spoken request. As we wont implement a semantic or contextual speech recognition we have to apply regular expression in order to attempt to find the key elements such as fligh number and date. The human requests the information in dozens of possible variations plus the varying interpretations by the speech-to-text engine, some listed below in the regex test screen (link).
To keep it simple we allow the user to request flight info for one particular British Airways flight number between yesterday and tomorrow. We will look for a 2 character 1..4 number combination in the text, plus using the keyword yesterday/tomorrow/today (or no statement representing today).

regular expression to identify flight number

To push the scenario further we can let the TTS engine read the response to the user, after selecting relevant information from the JSON formatted WS response.

Soundsamples

Sourcecode

References