Open the Box Workshop

Table of Contents

1      Anatomy of an Open the Box Application
1.1         Presentation
1.2         Prerequisites
1.3         Application Metadata Description
1.4         AppStore Applications Eligibility
1.5         Web Interface for Configuration / Use
1.6         Device Management
2      Life Cycle of an Open the Box Application
2.1         Development
2.2         Production
2.3         Dashboard Use
3      Application Example: Heating Control System Application
3.1         Application Structure
3.2         Device Access
3.3         Application Code
3.4         Application GUI
4      Base Driver Example: Heating Base Driver
5      References & Tutorials
5.1         Conditional Permission Admin
5.2         Device Access Specification
5.3         Declarative Services
5.4         Maven
5.5         Other Examples

1         Anatomy of an Open the Box Application

1.1       Presentation

Open the Box applications are programs intended for running on a Java OSGi platform on which management services pertaining to the Open the Box model were added,  gathered with name HABAM ; these services manage the installation of applications (locally or remotely), the access permissions to devices, the resource management and interfaces towards a display application (dashboard). An Open the Box application can be composed of one or more OSGi bundles, along with a set of metadata that are necessary for its integration into the AppStore and the management platform (PFG).

This document requires from the reader a minimal knowledge of the OSGi ecosystem and technology. However, in order to improve OSGi skills, references to tutorials about the different technologies are given in the annex (see section 5).

For a basic presentation of Open the Box concepts, see here.

1.2       Prerequisites

Currently, we use the Knopflerfish OSGi framework, release 4.0.1 (the last production release of Knopflerfish for OSGi R4), with a Java virtual machine in version 7 (Oracle implementation), however the code of HABAM services is developed so as to remain compatible with Java ME CDC 1.1. It all runs notably on the Myriad JBed JVM with Java ME CDC Foundation Profile on a Livebox Pro V3.

The OSGi platform hosting the application must a minima supply the following services:

• The OSGi Log service.
• The OSGi Http service for the application’s configuration and the use of web interfaces.
• An XML parser: the current implementation uses the xmlpull interfaces and therefore requires that an implementation of these interfaces be available (for example the kxml bundle, already present by default in Knopflerfish as it is used by Declarative Services). The parser is used for analyzing the dependencies in description files in terms of devices.
• The OSGi service Conditional Permission Admin (see § 5.1).
• Declarative Services specification  (see § 5.3): this is the format that applications must use to deal with searching and binding to the different OSGi services they need.

1.3       Application Metadata Description

Some metadata are necessary for a proper integration of applications into the Application Store (AS) and into the Open the Box dashboard:

1.  The application logo (size 72×72 pixels): it must be a file inside the bundle jar, whose path is given in the Manifest, field “Application-Icon”. If the logo is not supplied, a default one will be used. Example : Application-Icon: OSGI-INF/logo.png (corresponds to a file ~/workspace/<project name>/src/main/resources/OSGI-INF/logo.png).
2. The name of the provider, stated in the Manifest, field “Bundle-Vendor”. In order to use the service providers’ administration page, the value must be a name that is authenticated by the PFG, for example : Bundle-Vendor: hackathon
3. Information describing the application: label, descriptions, price, etc.

Note : if the jar is built through Maven, it is possible to define the fields Application-Icon and Bundle-Vendor in the instructions of the maven-bundle-plugin :

• <Application-Icon>OSGI-INF/logo.png</Application-Icon>
• <Bundle-Vendor>hackathon</Bundle-Vendor>

Information on the application are given in the file OSGI-INF/otb.properties that must be in the application bundle (if the application is composed of several bundles, one and only one of these bundles should contain this file). The file otb.properties must be encoded in UTF-8 and respect the classical Java syntax for properties files (cf. java.util.Properties).

The template of file otb.properties is as follows:

# Open the Box Properties

# otb.label: title of the application on the AS and on the Dashboard
# Optional but strongly advised (by default, field Bundle-Name of the manifest)

# otb.config.url: URL of the application web page.
# Must be exactly the alias of the servlet as registered to the OSGi http service.

# otb.category: list of items to select among: 
#    bureautique, domestique, education, jeux, multimedia, social, utilitaires.
# Optional.

# otb.description: simplified description of the application on the AS (short, one-line sentence).
# Optional but strongly advised (by default, empty string)

# otb.fullDescription: full description of the application on the AS (paragraph).
# Optional but strongly advised (by default, empty string)

# otb.price: application's selling price. Only for display, purchase is simulated!
# Optional. By default : 0,00

Example:

# Open the Box properties
otb.category=domestique, utilitaires
otb.label=Heating System
otb.description=Control the heating system in your Smart Home!
otb.fullDescription=Control the heating system in your Smart Home! \
   <b><i>Heating System</i></b> is an application designed for your \
   Smart Home Box which controls the heating system in your whole house.
otb.config.url=/app/MyAppUrl
otb.price=5,00

1.4       AppStore Applications Eligibility

When a user browses the AS from the dashboard, visible applications are the eligible ones, i.e. applications that are potentially installable in her/his environment. The eligibility computing is based on the references to devices in the Declarative Services configuration file (file with a name *devices.xml that must be in folder OSGI-INF).

An application is eligible if all its references are eligible.

Is taken into account:

• The interface, that must correspond either to a service, or to a device that is referenced and present on the customer’s HAB.
• Cardinality: 0..1, 0..n, 1..1 or 1..n. In cases 0..1 and 0..n, the reference is automatically considered as eligible, otherwise we check the presence of at least one current device that corresponds to the interface ; no distinction is made between 1..1 and 1..n, and 1..2 or 2..n cardinalities are not allowed, we do not count the exact number of devices but only their presence / absence.
• Filtering, that allows a finer description of the targeted device, via its serial number or its “friendly name”.

Example : target="(DEVICE_FRIENDLY_NAME=heater .*)"

Accepted keywords are: DEVICE_FRIENDLY_NAME, DEVICE_DESCRIPTION, DEVICE_SERIAL, SERVICE_PID. Multiple conditions can be chained with && or ||. Example:

<reference name="device" cardinality="1..n"
   interface="com.orange.dhi.otb.mocked.osgizigbeedevice.api.MockedOsgiZigBeeDeviceApi"
   target="(DEVICE_FRIENDLY_NAME=heater .*) || (DEVICE_SERIAL=xxx)" />

Properties respectively correspond to:

• DEVICE_FRIENDLY_NAME: property brought by the Base Drivers, trying to give the most vivid description of the device.
• DEVICE_DESCRIPTION, DEVICE_SERIAL: properties defined in the OSGi Device specification.
• SERVICE_PID: OSGi service id.

1.5       Web Interface for Configuration / Use

All applications must publish a web interface (http) that supplies at least the configuration and/or use of the application. A link to this interface is displayed on the dashboard, from the application’s detailed page.

The url of this page is specified in the application’s metadata (otb.properties file, see section 1.3). As this interface is served by the HAB, this url is a relative address that corresponds, in practice, to the registration alias from the application’s servlet to the HTTP service.

This web interface may be built any way, so long as it produces a browsable page at the specified address.  A flexible and efficient approach, recommended but not compulsory, is to implement a REST API (for instance JSON on HTTP) and to serve a static HTML page using this API. This approach is implemented in the demo application and presents the following advantages:

• The GUI (HTML + JavaScript to call the API) can be developed outside the OSGi framework, which turns out to be very useful because the latter just provides the HTTP service and does not support any  high level tool for web development.
• The REST API may be used by any other application (native Android / Ios application for example, or an application for Smart TV, Set top box, etc.) without modifying the code.

1.6       Device Management

The devices connected to the HAB are represented as OSGi services published by the Base Driver that corresponds to the device technology (ZigBee, EnOcean, etc.), complying with OSGi Device Access specification (see § 5.2).

An Open the Box application uses devices by accessing to their services and their methods. It can also subscribe to events by registering so-called “Service Listener” objects (e.g. ZigBeeEventListener for ZigBee technology) according to the White Board Pattern, or “Event Handlers” objects (Publish-Subscribe paradigm, e.g. used by the EnOcean Base Driver). Open the Box access rights management applies on method calls to services that represent devices. Access rights management to Service Listeners and potential Event Handlers using Event Admin is not supported.

Access to services that represent devices is controlled via the OSGi Conditional Permission standard mechanism. This means that an application may use, and see, only the services that represent the devices for which it has been explicitly granted the permissions. These permissions are granted and removed via the OTBAccessControlManagement service, an HABAM API (only accessible to the framework operator bundles, and not to standard user applications) or via the HAB dashboard (that, for itself, can request HABAM APIs).

An application describes its needs for devices through a file OSGI-INF/*devices.xml, with format specified by Declarative Services.

A dependency towards a device is defined by several elements (see § 1.4):

• An interface : this is the Java interface, as registered by the service that represents the requested device (registration is performed by the Base Driver).
• A cardinality (by default 1..1).
• A filter (optional) for specifying the targeted device(s), and the conditions on the properties of the service that represents the device.

Example of dependency:

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0"    
      name="HeatingApplication" immediate="true">
   <implementation class = "com.orange.openthebox.hab.heatingcontrolapplication.HeatingControlApplicationImpl" />
   <reference name="log" cardinality="1..1" interface="org.osgi.service.log.LogService"
      bind="setLog" unbind="unsetLog" />
   <reference name="httpService" cardinality="1..1" interface="org.osgi.service.http.HttpService"
      bind="setHttpService" unbind="unsetHttpService" />
   <reference name="heaterService" cardinality="0..n" policy="dynamic"
      interface="com.orange.openthebox.hab.heatingsystemdriver.Heater"
      bind="setHeaterDevice" unbind="unsetHeaterDevice"
      target="(DEVICE_FRIENDLY_NAME=Heater ?)" >
</scr:component>

For further details on the different types of connectivities used by devices, see chapter Open the Box Device Management.

2         Life Cycle of an Open the Box Application

This section describes how to test and make available to users an Open the Box application.

In a first stage, during the development / debug phase, we will show how to deploy the application locally – on the platform – and test it, and then once the application is stabilized, how to upload it on the AS.

Figure 1 Servlet HABAM

2.1       Development

2.1.1         Build

We will not precise here which programming environment the Open the Box application developer must use to write her/his code (even if we advocate Eclipse + Maven), we will just suppose that it ends up building a jar file that is a bundle in the OSGi sense.

Reminder: we supply here a complete development kit around these tools.

Once the compilation is correct in Eclipse, from a terminal, just:

• go in the project folder (e.g. cd ~/workspace/hackathon.template.app)
• synchronise Eclipse and Maven environments: mvn eclipse:eclipse
• launch the build : mvn clean install
• the bundle is created in directory ~/workspace/hackathon.template.app/target : hackathon.template.app-1.0.3-SNAPSHOT.jar (warning: the jar name depends in fact from the pom.xml, by default fields artifactId and version)

2.1.2         Deployment

Open the Box offers, on top of the OSGi granular bundle deploy model, an applications deploy model. This model specifies informations files necessary for building the AS and for managing customer applications on the Smart Home operator’s platform. This includes the declaration of access rights for the application on devices, thereby computing eligibility of applications for the client par the AS.

OSGi’s classical deployment mode (life cycle and commands install, update, uninstall) is replaced here by the Open the Box applications deployment mode. This deployment manages access permissions to technical services, in particular to devices.

Applications can then be deployed with two methods.

–          In development phase: locally deploy the application through an administration web page available on the box. This method is more efficicent during the development and test cycle, and has the advantage of not making public the applications throughout this phase.

–          In production phase: install the application from the AS, which supposes that it was previously uploaded there, hence made public. This method is therefore reserved to production applications, that must have been tested before.

The local administration web page is a servlet that comes with HABAM services, it is therefore started with Knopflerfish platform. It is usable from any computer connected on the same local network as the Raspberry, hence from the programmer’s environment. Its URL is:

http://<@IP-target:port>/habam/index.html, for example http://10.0.10.1n:8081/habam/index.html

The IP address is Raspberry target’s one, port being defined in configuration file props.xargs (-Forg.osgi.service.http.port=8081).

The home page (see Figure 1) presents a list of HTML forms that correspond to servlet actions. We will not detail here all these actions, very intuitive, but simply note that “Upload Application” is for uploading the Open the Box application’s bundle (~/workspace/hackathon.template.app/target/<bundle name>-<bundle version>.jar) : if the ’installation went through, the page will return the created ’application’s ID, which will be subsequently used for other servlet operations (start, stop, uninstall, update, etc.) “Update Application” is for updating an Open the Box application, based on OSGi command “framework/update”, that is without desinstall / reinstall of the application.

2.1.3         Handling

Once installed, the application can be handled in two different ways: either from the dashboard (it appears on it after deployment is over), or via thes HABAM ’administration servlet . It is possible to start, stop or deinstall it, or to browse its configuration page.

An action that is available only on the dashboard allows the user to finely associate the application’s access rights to the different devices detected on the platform (by default, deployment grants the application with access rights on all services that were stated on the declarative services configuration).

2.1.4         Multi-bundle Applications

It is possible to create Open the Box applications that are composed of several OSGi bundles, on condition that only one among them contains a file otb.properties. The application is made of a zip file that contains at the root the jar files of the different bundles. Local deployment is made from the same administration page (methods upload /update accept jar and zip files).

2.2       Production

Remember that local deployment as described above is not synchronized with the PFG. The applications installed in this way are neither seen from the PFG nor from the ’AS.

Once considered complete and stabilized enough for a public deployment, it becomes possible to publish it on the AS.

This process uses an API that is made available on Internet by the PFG. The URL of the ’administration page for service providers is: http://ohomeint.orange-labs.fr:9000/OpenTheBox/sp/index.jsp

Connection login/password: hackathon/hackathon.

The tab “Application/Submit” gives a way to upload the application jar (or zip). It will then be available on the AS, more precisely visible from all HABs on which it is eligible.

2.3       Dashboard Use

For configuring the Open the Box dashboard, see here.

2.3.1         Applications Local Management

Once an application is installed, either locally (see § 2.1.2) or from the AS (see § 2.3.2 under), it is possible to drive it locally from the dashboard: a button (with the application’s logo, as defined in the metadata, or a default icon) opens a generic page for the ’application. By default, during installation, the framework grants the application with access permissions to the devices it explicitely references.

Figure 2 Dashboard Home Page

• Click on button “Manage permissions” (see Figure 3). The dashboard, via le Dashboard Server, queries HABAM for the list of connected devices, then selects those that are compatible with required services that are referenced by the application. These devices are displayed with a tick-box: the user selects them, and clicks on “Apply”.
  • Chosen devices are associated with the application, i.e. access rights are set.
  • If devices have been unticked, corresponding access rights  are removed from the application.
• Click on button “Configure”. This will bring the browser on the application’s configuration page. This page is functionally absolutely independant from the HABAM framework and only related to the application.
• Click on “Start” button to launch the application, “Stop” to stop it.
• To deinstall the application, click on the trash bin icon lower left of the dashboard. Un confirmation dialog opens up to avoid errors. We must precise that when an application is deinstalled from a HAB (either locally or from the AS), it is possible to re-install it from the AS.

Figure 3 Permission Management

2.3.2         Handling Applications from the AS

From the tablet, display the AppStore home page (url WAN + authentification).

• The AS displays the applications it can install on the HAB (applications “eligible” according to present devices).

  

  Figure 4 Application Store

• Click on the selected application.
• On the tablet screen, we now have a page that is related to the application (containing a description and the list of required services) with a “Select” button for simulating purchase. Click on it, a new “Install” button appears. Click on this button.
  • The application is installed on the HAB.

  

  Figure 5 Installation of an application

• The application is installed and automatically started, a “Stop” green button appears. By default, HABAM grants the application permissions access to all explicitely referenced services in the Declarative Services configuration  file (see Device Management). To modify this cette distribution (remove access rights to some devices, or add them if new devices were installed after the application started), dashboard must be used (cf § 2.3.1).
• Click on “Stop” button.
  • The application stops.
  • The Stop button is replaced by a “Start” red button.
  • An “Uninstall” button allows the user to deinstall the application from the HAB.
• Click on “Start” button.
  • The application starts if correctly configured.
  • The Start button is replaced by a green “Stop”.

3         Application Example: Heating Control System Application

The application Heating Control System ( hackathon.template.app.zip) is a dummy application for heating system control that aims to give the principles for developping a new application for the open the box Framework. The main points are as follows:

• based on Declarative Services to get the references of services reifying the device used by the application (heaters and thermometers) ;
• REST API publication for application control;
• presentation of a Web API based on static html code using the previous REST API.

Functionnally, the application is rather simple, it does nothing except displayiong the list of equipments that it access to. An interesting extension (let to the reader as a trainning) consists in implementing a fedback control loop from the temperature reported by the thermometer(s) to the heater(s) control through its(their) on/off command.

The displayed equiments, Heater(s) and thermometer(s) are virtual ones ! And their base drivers are installed in the next section.

We deliberatly choose to show a rather simple application. For a more complex example including ZigBee devices access see thistutoriol.

3.1  Application Structure

The application is implemented through two classes only : one component Declarative Services et one Servlet.

 

3.2  Equipements Access

Equipment access uses Declarative Services. Application dependencies to equipments is declared in the html file src/main/resources/OSGI-INF/component-devices.xml :

   <?xml version="1.0" encoding="UTF-8"?>
   <scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.2.0" name="HeatingController" immediate="false">
      <implementation class= "com.orange.openthebox.hab.heatingcontrolapplication.HeatingControlController" />
      <reference name="log" cardinality="1..1" interface="org.osgi.service.log.LogService"
         bind="setLog" unbind="unsetLog" />
      <reference name="httpService" cardinality="1..1" interface="org.osgi.service.http.HttpService"
         bind="setHttpService" unbind="unsetHttpService " />
      <reference name="heaterService" cardinality="0..n" policy="dynamic"
         interface="com.orange.openthebox.hab.heatingsystemdriver.Heater"
         target="(DEVICE_FRIENDLY_NAME=Heater *)"
         bind="setHeaterDevice" unbind="unsetHeaterDevice" />
      <reference name="thermometerService" cardinality="0..n" policy="dynamic" 
         interface="com.orange.openthebox.hab.heatingsystemdriver.Thermometer"
         bind="setThermometerDevice" unbind="unsetThermometerDevice" />
   </scr:component>

Grâce à Declarative services, The services offered by the equipments available for the application are bind ( keyword bind) and unbind ( keyword unbind) by the corresponding method (un)setXXX invokation (see above : (un)setThermometerDevice). If Permissions are modified, the application should be stoped then restarted to update its permissions. Conditional Permission does not take into account dynamically permission modification for a running bundle. .

3.3       Code de l’’application

The HeatingControlController Class, defined as Declarative Services component implementation, exports activate/deactivate/modified methods that are invoked by the framework to update the OSGi ComponentContext .

La référence au service OSGi HttpService permet d’’enregistrer la servlet.

3.4       IHM de l’’application

The APIServlet servlet exports a REST API under http (JSON coded). This API includes two resources that display the heater and thermoter lists to which the application can access :

• http://<ip_hab>:<port>/HeatingControl/api/heaters
• http://<ip_hab>:<port>/HeatingControl/api/thermometer

Then, the ’IHM is implemented by static files (in the bundle, static directory), used as simple ressources by the OSGi http service. Some ’API REST calls are implemented , through JavaScript, in order to get a dynamic IHM. It will be easy to develop an Android or iOS application using this API.

^THe IHM can be accessed at:

http://<ip_hab>:<port>/HeatingControl/static/index.html

4         Base Driver Example: Heating Base Driver

The Heating Control System application uses équipements that have being published by the HeatingSystem Driver base driver(sample.heatingsystem.driver.zip). This is in fact a virtualBase Driver that publishes services corresponding to two Equipment types : heaters and thermometers.

All these equipments are purely virtual : no communication towards some concrete equipments is carryed out, this virtual driver is nothing but a help to show how equipments are published by OSGi services. It particularly demonstrates how to register and how to unregister an OSGi service with all necessary properties for a proper integration in the Open the Box platform.

A “Best Practice” guide for developing base drivers in the ‘Open the Box framework can be found in a separate document.

5         Références & Tutoriels

5.1       Conditional Permission Admin

Conditional Permission Admin is the OSGi standard for permission management, It extends Permission Admin with the dynamic permission assignment while the OSGi Framework is running : there no need to stop and restart the OSGi Framework to take into account a new permission assignment for some application. At the contrary, because of the parametred permission type (e.g. package Java import rights) the bundle athat are impacted by these permission modification should be sttep then restarted (This is expecially the case for Open the Box applications ).

Conditional Permission Admin allows to define new permissions as rules with the following syntax :<access> <condition> <permission>

Par exemple :

ALLOW {
   [org.osgi.service.condpermadmin.BundleLocationCondition "file:/opt/kf/totest/*"]
      (org.osgi.framework.PackagePermission "*" "import")
}

Où :

• Access : ALLOW
• Condition : org.osgi.service.condpermadmin.BundleLocationCondition "file:/opt/kf/totest/*"
• Permission : (org.osgi.framework.PackagePermission "*" "import")

References :

• Specification : chap. 9 des Core Specifications : http://www.osgi.org/download/r4v42/r4.core.pdf
• Tutorial and examples : http://www.knopflerfish.org/releases/4.0/docs/osgi_with_security.html

Rappel :

• We use a corrected (and thus improved) version of the Knopflerfish framework for permission management, see notes from Installation Guide.

5.2       Device Access Specification

The Device Access specification defines how to use OSGi services in order to represent home equipments within an OSGi framework , and where the base drivers are localized to allow the publication of these services.

The best exemple of Device Access use is the UPnP base driver specification.

Références :

•  Service Compendium specification, chapter 103 : http://www.osgi.org/download/r4v40/r4.cmpn.pdf
• the UPnP base driver specification: Service Compendium specification, chapter 111 : : http://www.osgi.org/download/r4v40/r4.cmpn.pdf

5.3       Declarative Services

Declarative Service uses « dependancies injection » mechanism that prevent from having to manage the service search as well as dependancies life-cycle. The services used by some bundle code are declared un meta-data file xml coded and the corresponding instances are straightforwardly given to the component at run time, as soon as these declared services become available.

Références :

• Injection de dépendances : http://en.wikipedia.org/wiki/Dependency_injection
• Service Compendium specification, chapter 112 : http://www.osgi.org/download/r4v40/r4.cmpn.pdf
• Tutoriel :
  • Sur le wiki OSGi : http://wiki.osgi.org/wiki/Declarative_Services
  • Tutoriel Felix (applicable directement avec Knopflerfish) : http://felix.apache.org/documentation/subprojects/apache-felix-service-component-runtime.html
  • The Heating Control System (voir supra) demonstration application is also a good starting point for the use of Declarative Services

5.4       Maven

Maven is a very popular Java build system based on a declarative style (“What we want to get” is given as inputs but not “how to get it”). Maven also supports dependencies management.

For OSGi bundle development we recommand the use of Maven together with the Maven bundle plugin which embeds the bnd (http://www.aqute.biz/Bnd/Bnd) tool for including this development in a Maven build >.

Maven bundle plugin : http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html

To get further insight in the use of Maven, we highly recommanded the reading of both the base driver and the exemple application Heating Control given in this present document;

5.5       other examples

.

Other Open the Box application exemples are included in tutoriel : e.g. an application that manages ZigBee devices. Finally, for a quick kick off, an application template could be used.