MOOC – E-Learning on Steroids

Hardly any industry is moving as fast as the IT industry. While your operational experience and knowledge of the vertical domain you are working in, is growing naturally along your career, it is not the same for IT. For example the airport environment, the underlying basics and physics of handling aircrafts, planning flights, etc. are exposed to changes, innovations and challenges, look at the A-CDM program, it took quite some years to take off and become main-stream, that is a much slower than any new general IT technology or platform soaring. Though this industry is picking up speed too and the boundaries between the digital and physical world start to blur more and more, airports are running digital transformation programs, though passengers still flying in the physical world.

But on the IT side of things, the speed is way beyond breath-taking and it is hard to keep a minimal overview over many areas of IT concerns as well dive into specific topics. How to stay up-to-date and tune into relevant topics ? Books (ink and electronic versions) and forums are certainly the traditional approach, on top of that you join conferences and in-persons seminars and training (which comes at a cost and time spent).

Since the 2000’s online courses came into the picture, as the successor for e-learning, and allow a much bigger audience to learn new technologies, skills and more. The very positive part, there are lots and lots of free courses, most platforms offer free and commercial courses, sometimes free to participate and only charge a fee if you want to get an official certificate (one can argue about the value of such certs) but most important, you can learn and move forward and update your knowledge with the click of a button.

The big challenge though is to identify what you need or interested in, find the right courses and, most important, manage your time. Using your spare time you have to choose wisely, you can’t run for every course out there, even they are so many you are interested in and you are temped to sign up for a dozen of courses, only not to finish any of them.

Todays key-/buzzword for this is MOOC or Massive Open Online Courses. This is like e-learning on steroids, in the past you had to look at dull corporate slides pretty much by yourself, now we look at videos, reading material online and offline, interactions with the organizer, mentor, trainer or your virtual peers at various levels.

Not only the organizations that started online learning, like schools and universities, are into the game, as well companies operating specific online course platforms and now book publishing companies offering courses and finally professional social platforms like LinkedIn.

I attended online courses at Coursera and Udacity, which offer a broad range of topics, and now started with some specific courses on HCI and UX at Interaction Design Foundation which solely offers courses on UX, HCI, Visualization and related topics. Though the courses are unattended (except the rating of your text answers or comments) but repeating, you still have a motivation to participate and go though the lessons because you pay money and they help you pacing the whole course by releasing the lesson packages over time.

Stay tuned for the results.

Advertisements

Build a RESTful Webservice in less than 5 minutes

There are quite some tutorials around about building and exposing a RESTful Webservice, but some of them are outdated and make you wade through complex dependencies and tinkering with deployment descriptors and web.xml files. But using RESTeasy, the JBoss implementation that is fully compliant with the JAX-RS 2.0 JCP specification, and Eclipse you can build a simple webservice (“hello world”) with less than 10 lines of sourceode with annotations and no web.xml used in a few minutes and run it on Wildfly.

Lets build a webservice that creates random numbers.

Continue reading

Geolocation of mobile phones in the GSM network

As user of mobile phones we are used to have an almost 100% coverage for phone calls (not for data though) and the user-experience is absolutely seamless in most urban and sub-urban areas in Europe and most other countries. Moving around in trains and cars we cant sense the hand-over in the background of the GSM (2G, GPRS, EDGE), UMTS (3G), LTE (4G) network passing on our connection between the BTS ( base transceiver station). As regular user we dont have an idea about the number and location of cell towers around us, some towers or antennas are mounted on very obvious structures (antennas on towers), some are almost hidden. Though the GSM antennas can have a range of up to 35km (flat plane vs less than 5km in hilly areas), we have much higher density of cells in the urban area with antennas almost every few 100 metres or less. There are a lot of parameters influencing the infrastructure and its layout at a certain place, I wont dive into the details of if, you can get some info on the reference sites listed at the end of the article.

Rather approaching this topic hands-on, I was curious about the information that I can retrieve with Android about the active cells, its location and ultimately about the information the network operator (or other interested parties) collects about one. We might disable the GPS function of a phone to stop apps to collect our whereabouts, giving apps access to the phone state still gives a coarse location profile.

Usually phones dont reveal any network information other than the network operator but with the help of some regular Android methods of the TelephonyManager and GsmCellLocation class we get the crucial information.

The key info we are looking for is

  • MCC – Mobile Country Code
  • MNC – Mobile Network Code
  • LAC – Location Area Code
  • CELLID – The ID of the cell

Only the combination of the above 4 values is unique and can identify the location. You can look at a directory of MCC codes at Wikipedia, but there is no list of of LAC and CELLID codes published by the provider. But in the era of the “crowd” there is a collaborative community collecting the measurements of cellphones and putting them into a DB (CC-BY-SA 3.0). At  opencellid.org you can both retrieve information about cell towers as well download the complete DB.

OpenCellID.org - cell info

OpenCellID.org – cell info

I was curious how often we change the cell and what geo information I could retrieve from this, respectively build a geo profile of myself moving around. I build an app the listens to the currently active cell, lists it up on the screen and logs it into a csv files, and optionally gives an acoustic notification (beep) every time cell has changed.

MyCellDroid Beta App

MyCellDroid Beta App

Quite surprising, during one of the first tests, driving a 100km distance along the highway, both rural and suburban area, I passed through more than 80 cells !

Relevant Android methods

TelephonyManager telephonyManager = (TelephonyManager) this.getSystemService(Context.TELEPHONY_SERVICE);
mcc = telephonyManager.getNetworkOperator().substring(0, 3);
mnc = telephonyManager.getNetworkOperator().substring(3);
operator = telephonyManager.getNetworkOperatorName();

GsmCellLocation location = (GsmCellLocation) telephonyManager.getCellLocation();
lac = location.getLac();
cellid = location.getCid();

Be aware of the permissions required

 <uses-permission android:name="android.permission.READ_PHONE_STATE" />
 <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />

A extra note about Android 6.x+ development, Google changed massively the permission concept and apps require an additional confirmation of the required permissions (for so-called dangerous permissions) during runtime, this has to be implemented specifically, otherwise your application which runs on earlier Android versions will crash. I will share the relevant implementation in an upcoming entry.

Note for Samsung Phones: The function to get information about the nearby cells is not supported by Samsung phones

List<NeighboringCellInfo> neighborCells = telephonyManager.getNeighboringCellInfo();

This list will all be null on Samsung phones. You only can retrieve information about the currently connected cell.

I will try to make some more sense out of the collected data and see how fine-grain the collected data reveals my location.

Btw, there are dozens of similar apps in the Playstore and some even report back collected data to improve and build up the opencellid project database.

References:

  1. https://en.wikipedia.org/wiki/Cell_site
  2. https://en.wikipedia.org/wiki/Base_transceiver_station
  3. https://en.wikipedia.org/wiki/Sector_antenna
  4. https://en.wikipedia.org/wiki/Cellular_network
  5. https://en.wikipedia.org/wiki/Mobile_country_code
  6. https://developer.android.com/reference/android/telephony/NeighboringCellInfo.html
  7. https://developer.android.com/reference/android/telephony/TelephonyManager.html
  8. https://developer.android.com/reference/android/telephony/gsm/GsmCellLocation.html
  9. https://developer.android.com/guide/topics/security/permissions.html

Exploring a SQLite Database on Android

or “How to read SQLite DB from a desktop”

SQLite is the relational, embedded, ACID compliant database that comes with Android. Due to this fact it is certainly the most deployed DB engine on this planet. In case your application need to have CRUD features for local persisted data and the complexity level is beyond a simple text file, you have to consider it.

A challenge is to look into the (raw) DB from your desktop (if you dont want to build and integrate a DB viewer into your app). As Android apps store databases into their respective /data subfolder and if you don’t have a rooted phone, you cant look inside this folder.

I am not aware of any tool that can open a connection to the DB remotely, so the best way is to copy the DB file into the accessible SD card (or whatever the phone and its manufacturer considers as SD card, even the internal memory mounted as SD card), download it to your desktop and open it with a tool like the SQLite DB Browser.

Let’s put some sourcecode here as reference

Create a simple demo DB

No bells and whistles, no helper classes, etc. just the most simple way to create DB and a table.

 
    private void createDB() {
        SQLiteDatabase sampleDB = this.openOrCreateDatabase("MYDEMODB", MODE_PRIVATE, null);
        sampleDB.execSQL("CREATE TABLE IF NOT EXISTS MYTABLE (Last VARCHAR, First VARCHAR, Role VARCHAR);");
        sampleDB.execSQL("INSERT INTO MYTABLE Values ('Smith','John','CEO');");
        sampleDB.execSQL("INSERT INTO MYTABLE Values ('Thomson','Allan','CTO');");
        sampleDB.close();
    }

Copy the DB to your SD card

 
    private void exportDB() {
        File mySd = Environment.getExternalStorageDirectory();
        File myData = Environment.getDataDirectory();

        FileChannel src = null;
        FileChannel des = null;
        String currentDBPath = "/data/" + getApplicationContext().getPackageName() + "/databases/MYDEMODB";
        String exportDBPath = "MYDEMODB";

        File currentDB = new File(myData, currentDBPath);
        File backupDB = new File(mySd, exportDBPath);
        try {
            src = new FileInputStream(currentDB).getChannel();
            des = new FileOutputStream(backupDB).getChannel();
            des.transferFrom(src, 0, src.size());
            src.close();
            des.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

After download to your local drive you can use the SQLite Browser to open the file. Very useful data debugging or for apps that collect data and you can’t implement an upload of the data to a server via the internet connection.

SQLite DB Browser

SQLite DB Browser (Structure View)

 

SQLite DB Browser

SQLite DB Browser (Data View)

Hide Glassfish Server Information

Despite moving on to JBoss progressively I still share my findings, often they apply to other products too.

For a public facing server you want to reveal as little background information as possible. In Glassfish you can hide create custom 404 and 500 error pages (previous post), but you should also hide the server info that comes with the server header, easily revealed by a tool ike the Firefox plugin httpfox.

Response Header

Response Header

There are 2 crucial settings you must change:

  • JVM setting for product name: -Dproduct.name=”My App Server”

    JVM Settings

    JVM Settings

  • Remove the “XPowered By” flag

    XPowered By

    XPowered By

As result you will have a pretty generic response header

Response Header

Response Header

Increase size and type of AWS EBS volume

I was offline for quite a while because shifting from one continent to another. But now regular posts should be rolling in again.

I am running a couple of instances in pre-production requirement mode and changed from a standard EBS volume to a IOPS volume for the DB instance or the volume with the DB files. I could not identify a reasonable increase of performance, maybe a misconception that IOPS volumes will boost performance, rather provide a defined and consistent random access I/O throughput. I must admit I did not use a value higher than 1000.

Billing IOPS

Billing IOPS

Some recommended reading:

I decided to return to a standard ESB volume for my database as its performance did not benefit from the IOPS type (the DB is not overly busy too).
You cant change type and size of an EBS volume on the fly.

Here the steps to achieve the same: Continue reading

Copy EC2 instance to another region

Is it finally possible ? While the AMI import tool is long awaited for but only available for Windows, it is rather a big hazzle to transfer manually (see this) any other OS ( my last attempt in 2010).

Today Amazon announced the EBS Snapshot Copy Feature (across regions). The intention is certainly to allow easy migration of data to another region, as you can copy the snapshot, create a volume and attach it to an instance. I was curious to try if I can migrate my Ubuntu instance to another region and it worked. You can use both command-line as well the AWS web admin.