Injecting external configuration into Spring Boot and Docker

First, in order to use the configuration from an application.yml oder application.properties, its handy to abstract that with a Spring Component class.

@Component
@ConfigurationProperties("address")
public class AddressConfiguration {
    private String city;
    private String street;

    public AddressConfiguration(String city, String street) {
        this.city = city;
        this.street = street;
    }

    public String getCity() {
        return city;
    }

    public String getStreet() {
        return street;
    }

    public void setCity(String city) {
        this.city = city;
    }

    public void setStreet(String street) {
        this.street = street;
    }
}

The above class maps to the following application.yml:

address:
  city: "New York"

The name that is specified for the ConfigurationProperties annotation must match with the root element name of your configuration, that you want to load. As you can see, the application.yml doesn't contain an entry for street. You can set additional entries (or overwrite existing ones) by supplying a SPRING_APPLICATION_JSON environment variable:

export SPRING_APPLICATION_JSON='{"address":{"street":"Broadway"}}'

If you are running Spring Boot inside a Docker container, you can pass the exported SPRING_APPLICATION_JSON environment variable to the container by providing the -e flag:

docker run -e SPRING_APPLICATION_JSON="$SPRING_APPLICATION_JSON"

Raspbian Edimax USB WLAN Adapter

In order to get the Edimax USB WLAN Adapter working on Debian / Raspbian, proceed as follows.

(1) Use lsusb to list all USB devices:

$ lsusb

 Bus 001 Device 004: ID 7392:7811 Edimax Technology Co., Ltd EW-7811Un 802.11n Wireless Adapter [Realtek RTL8188CUS]

The Edimax USB WLAN Adapter uses the Realtek RTL8188CUS chipset.

(2) To figure out which driver is used and hence which module needs to be loaded, we can use the system message log:

$ dmesg | grep rtl

[   16.919884] usbcore: registered new interface driver rtl8192cu

Now we know the module name: 8192cu

(3) To load the module during system boot, create the following file with the given content:

$ sudo touch /etc/modprobe.d/8192cu.conf
$ sudo echo "options 8192cu rtw_power_mgnt=0 rtw_enusbss=0" > /etc/modprobe.d/8192cu.conf

(4) Enable the wlan0 device (eth0 will be disabled with this configuration):

#auto eth0
#iface eth0 inet dhcp
 
auto wlan0
iface wlan0 inet dhcp
wpa-ap-scan 1
wpa-scan-ssid 1
wpa-ssid "your-ssid"
wpa-psk "your-wlan-passphrase"

(5) Reboot your PI

Install VirtualBox on Debian Stretch 9.5

Prerequisites

Add the backports repository to /etc/apt/sources.list:

deb http://ftp.de.debian.org/debian/ stretch-backports main contrib
deb-src http://ftp.de.debian.org/debian/ stretch-backports main contrib

Installation

$ apt-get update
$ apt-get install virtualbox
$ apt-get install virtualbox-ext-pack
$ apt-get install linux-headers-$(uname -r)

Build Kernel Module

$ dpkg-reconfigure virtualbox-dkms

$ dpkg-reconfigure virtualbox

Links

https://wiki.debian.org/VirtualBox#Debian_9_.22Stretch.22

Cleaning up Docker containers with xargs

The Linux/UNIX tools landscape is always worth investigating, while you're trying to find a solution for a problem. Because, if you're lucky you don't have to write a script or anything similar: you can just chain existing tools together to build up your custom solution.

Thanks to Dump Tiger and Bjørn, I got to know the xargs command.

One thing that I don't like so much, when I'm starting/stopping Docker containers quite often, is that they'll polute your disc space (depending on the image size).

docker ps -a | grep 'Exited (' | cut -d ' ' -f1 | xargs docker rm

The pipe shown above does the following:

1. List all Docker containers
2. Filter the list, so that only those lines that contain the string 'Exited (' will be returned
3. From all returned lines, cut of the first row (the container ID) by splitting each line on all whitespaces
4. Now that we've a list of container IDs, we can pipe each container ID into docker rm by using xargs

Debian: OpenVPN Client Config

Getting your OpenVPN client up and running on Debian is easy.

Prerequisites: The openvpn.conf file from your VPN provider.

1. apt-get instal openvpn
2. apt-get install resolvconf
3. Make sure the following settings are enabled in your openvpn.conf:
• script-security 2
• up /etc/openvpn/update-resolv-conf
• down /etc/openvpn/update-resolv-conf
4. openvpn --config /path/to/openvpn.conf

Gradle Recipe: Building a Fat JAR

So far, I haven't seen a Build Tool that is easy to use. Doesn't matter if it was called Make, Maven or SBT. They're all very complex and far away from easy and intuitive.

Since I'm struggling a lot with Gradle lately, this is the first post that shows how I solved my (common) problem: Building a Fat JAR.

I don't know if I see things differently, but my first expectation on a Build Tool is, that it builds my software with all the configured dependencies. Therefore, when using Gradle, you can choose between three dependency types:

• compile
• runtime
• testCompile

As soon as you add a new dependency, you would probably declare it as a compile dependency, because you need it to be able to compile your code. In case you are adding TestNG or JUnit, you declare it as testCompile, because you only need it for your tests. The runtime dependency extends from the compile dependency. My understanding of runtime would mean, that I need this dependency in order to be able to execute my code (for example on a production environment). The good thing is, that it means exactly this. The bad thing is, that it does not automatically pack all your dependencies together, so that you can ship your executable.

Lets say you use Apache Log4j2:

dependencies {

   runtime 'org.apache.logging.log4j:log4j-core:2.10.0'

}

From a user perspective, I would assume that this is everything I need to do - but its not. If you try to execute the JAR that is being build, you'll get a ClassNotFoundException.

The solution is the following build.gradle file:

apply plugin: 'java'

test {

   useTestNG()

}

repositories {

   mavenCentral()

}

dependencies {

   compile 'org.apache.logging.log4j:log4j-api:2.10.0'

   compile 'org.apache.logging.log4j:log4j-core:2.10.0'

   testCompile group: 'org.testng', name: 'testng', version: '6.13.1'

}

task fatJar(type: Jar) {

   baseName = project.name

   from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }

   with jar

   manifest {

      attributes(

         'Main-Class': 'org.tobster.foo.GradleExample'

      )

   }

}

assemble.finalizedBy fatJar

Since this is a very common problem, you'll find a lot of solutions, but this was the only one that worked for me. The most interesting part for me is, that you have to write a custom Gradle Task that assembles a Fat JAR. Why do I've to do that? Wouldn't it be possible to have some kind of flag, e.g. fatJAR=true. Maybe that would be to easy.

Data Visualization with Grafana and Elasticsearch

If you want to store and visualize data, you have a lot of technologies to choose from. Two such technologies are Elasticsearch (ES) and Grafana. Something that I really like about ES, is that its very easy to use and that it needs no complicated setup or configuration. It just works out-of-the-box (production environments may require more sophisticated configuration). Even though Logstash and Kibana are probably the most well known companions for ES, especially when it comes to visualization of logging data, Grafana is also quite good for time series based data and it keeps your technology stack simple. In the following, I will describe how to store data in Elasticsearch 5.6.0 through its HTTP API and visualize it with Grafana 4.4.3.

Elasticsearch

1. Download ES 5.6.0 from the official site and unpack
2. bin/elasticsearch
3. Web-Frontend can be found at http://localhost:9300

Elasticsearch is a document database that stores documents as JSON. Documents can be stored and retrieved by its HTTP API (there are also libraries for the most popular programming languages). Furthermore, ES stores documents into what is called indices and types. An index could be compared to a database in a relational DBMS and a type could be compared with a table.
For example: you could have an index called customers with a type invoices and another type addresses. You can think of types about something that logically belongs to an index (a customer has an address and one or more invoices).

Grafana

1. Download Grafana from the official site and unpack
2. bin/grafana-server
3. Web-Frontend can be found at http://localhost:3000

Configure a Data Source:

1. Go to http://localhost:3000/datasources and click on "add data source"
2. Enter a name for the data source and select "Elasticsearch" from the Type drop-down
3. As long as you didn't changed the default ES configuration, enter "http://localhost:9300" in the Url field.
4. Access must be set to "Proxy"
5. Enter the name of your index (for this example, we need no pattern)
6. Every JSON document you insert into ES must have an ISO date field, e.g. 2017-07-20T14:00:00. This is important, because as mentioned before, Grafana is designed to work with time series based data. Therefore, enter the name of the field that contains the ISO date into the field "time field name" (without a leading @). I called this field "insert_time" (as shown in the screenshot below).
7. Since we are using ES 5.6.0, select version 5.x from the Version drop-down

Configure a Dashboard:

1. Go to http://localhost:3000/?orgId=1 and click on "Create your first dashboard"
2. Click on "Graph" to create a new graph dashboard
3. Click on "Panel Title", then click on "Edit" (after that, the Graph configuration appears down below)
4. In the "General" tab, you can give the dashboard a name
5. In the top-right corner, select a time range (e.g. last 90 days), where you know that there is some data
6. If you wonder why you don't see anything, go to the "Display" tab and
• Draw Modes: select "Lines"
• Stacking & Null value: select "connected" for Null value
7. In case your JSON documents have a field that contains a numeric value that you want to visualize, you can also use the existing aggregate functions. Therefore, go to the "Metrics" tab and for example select "Average" as Metric and your timestamp/date field for "Group by" and select "Date Histogram" from the drop-down.

That's it! Thank you for reading.