DIY FreeNAS based system

Introduction

This time I won’t be writing about software related stuff but about building a Network-Attached Storage (e.g. NAS) based on FreeNAS.  A family member asked me if I would be able to build a custom NAS system for him.  As some years have already passed by since  my last PC build, I wasn’t up-to-date with the current available hardware and I didn’t have any knowledge of NAS setups what so ever.

The requirement were:

  • the data should be stored redundantly;
  • it should be easily recoverable in case of failure;
  • it should backup data to an external usb drive;
  • the system should be easy to maintain;
  • the system should have a low power consumption;
  • the system should be quiet;
  • there should be a notion of users and permissions with dedicated folder;
  • it should be able to recover unwanted modifications on data;
  • and if possible run a mail server on top of it.

Thanks to the help of some people, and especially CiPHER, at the tweakers.net forum, I was able to determine the setup I was going to assemble.

Components inventory

NAS PC Components

Here is the list of the components shown above:

Assembling the compontents

Here’s how I proceeded for assembling the NAS components:

  1. Open the side panel of the cooler master.
  2. Mount the 4 metal hex nuts, which are supplied with the mini-tower, so that they match the Mini ITX Form Factor on which the motherboard will be mounted.
  3. Put the I/O Panel Shield on the back of the motherboard and fix it on the case.
  4. Now we’ll put the 2 memory modules in the memory slots.
  5. Wire the picoPSU peripheral extension cable to the picoPSU.
  6. Wire the picoPSU on the ATX power connectors (24-pin and 4-pin connector) of the motherboard.
  7. Wire the cables of the case regarding power led, HD led, power switch and reset switch to the motherboard.
  8. Wire the front panel USB cable of the case to the motherboard.
  9. Wire the HD audio interface cable of the case to the motherboard.
  10. Mount the SSD in the floppy drive bay.
  11. Mount the 3 HD’s in the hard drive bays.

The result of it should look like the picture shown below.

Assembled PC Compents

Running rsync via cron job

One requirement was that a backup should be made from the ZFS pool offsite to an external USB drive. To achieve that I wrote a script based on an example that I found on the freenas forum.

  1. From what I understood on a blog about Freenas is that it loads all it’s stuff from the “/conf/base”-directory in a RAM disk and maps it to the standard Unix installation structure. This means that every modification you made will be lost after a reboot or shutdown. To make those modifications permanent, we have to mount the volume so that we can write on it. To do that execute the following in the shell:
    mount -uw /
  2. Now we are going to write a shell script containing the logic for rsync-ing the ZFS pool to an external USB drive:
    nano /conf/base/var/backup.sh
  3. Once in nano, or any other editor you whish to use, enter the following:
    #!/bin/bash
    partition=/dev/$1
    origin=/mnt/$2
    target=/mnt/backup
    
    backup() {
    	if [ -e ${partition} ]; then
    		/bin/mkdir -p ${target}
    		/bin/chmod 777 ${target}
    		/usr/bin/logger "Mounting USB Device named ${partition} to ${target}..."
    		/sbin/mount -t ext2fs ${partition} ${target}
    		/usr/bin/logger "Mounted USB Device named ${partition} to ${target}"
    		/usr/bin/logger "Rsyncing ${origin} to ${target}..."
    		rsync -av --progress ${origin} ${target}
    		/usr/bin/logger "Unmounting USB Device named ${partition} from ${target}..."
    		umount ${target}
    		/usr/bin/logger "Unmounted USB Device named ${partition} from ${target}"
    		/bin/rm -r ${target}
    	else
    		/usr/bin/logger "USB Device named ${device} not present"
    	fi
    }
    
    backup;
    
  4. Make sure the script is executable:
    chmod 777 /conf/base/var/backup.sh
  5. Set the base volume back to read-only:
    mount -ur /
  6. Exit the shell by typing “exit” in the shell and hit the reboot option.

Once the FreeNAS system is rebooted, you should see via the shell that the script is located under /var/backup.sh. Now let’s create a cron job via the web interface of FreeNAS. As you’ll could have noticed when typing the script, there are 2 arguments:

  • the first argument specifies the device name of the usb device (without the prefix /dev)
  • the second argument specifies the name of the ZFS pool (without the prefix /mnt)

Let’s consider as an example that the full path to the device would be “/dev/da1s1″ and the full path to the ZFS pool would be “/mnt/dablomatique”, the script have to be executed as follow:

/var/backup.sh da1s1 dablomatique

That’s it for the moment. If there are any suggestions, they are always welcome.

Building collapsed EAR through Maven for Weblogic & Tomcat with OpenEJB aka TomEE

Introduction

For this tutorial I had the challenge to deploy the same Java enterprise application (e.g. EAR-file) on Oracle Weblogic as well as on Apache Tomcat.

First of all you have to know that Tomcat isn’t an application server.  It also doesn’t support the deployment of EAR-files, which is logical considering the first statement.  Luckily, since JEE6, there is the possibility of the collapsed EAR technique (see http://openejb.apache.org/collapsed-ear.html), which consist of repackaging the EAR-file into WAR-file which is deployable on Tomcat.  Tomcat doesn’t also support Enterprise Java Beans out of the box, so that’s were TomEE comes to the rescue. TomEE is a project which simplifies the usage of OpenEJB on Tomcat.

In this article I will mainly focus on Tomcat and less on Weblogic because it was more challeging to make it run on Tomcat.  The result is that the application will run on both servers.

The application uses Java Persistence Api 1.0 with Eclipse Link to connect to the database.  This is all exposed through EJB’s.  There is also the need to talk to a JMS Queue and process the messages via a Message Driven Bean.

The UI is done with Java Server Faces 2.  The application has of course to be unit tested.

Used technologies

The application uses several JEE technologies, namely:

  • JPA or Java Persistence API 1.0 with Eclipse Link 1.1.0 as provider.
  • EJB or Enterprise Java Beans 3.0 with OpenEJB 3.1.4 as provider.
  • Bean Validation API 1.0 with Hibernate Validator 4.0.2 as provider.
  • JMS or Java Messaging Service with ActiveMQ 5.2 as provider.
  • JSF or Java Server Faces 2.0 with the Glassfish reference implemenation as provider.
  • JAX-WS or Java API for XML Web Services with Glassfish Metro and Apache CXF as possible providers.

Configuration in 10 steps

  1. Install Tomcat 6.0.35.
  2. Copy ojdbc14-10.2.0.4.0.jar to %TOMCAT_INSTALL_DIR%/lib.
  3. Copy eclipselink-1.1.0.jar to %TOMCAT_INSTALL_DIR%/lib.
  4. Copy validation-api-1.0.0.GA.jar to %TOMCAT_INSTALL_DIR%/lib.
  5. Copy hibernate-validator-4.0.2.GA.jar to %TOMCAT_INSTALL_DIR%/lib.
  6. Copy slf4j-api-1.6.1.jar to %TOMCAT_INSTALL_DIR%/lib.
  7. Copy openejb.war to %TOMCAT_INSTALL_DIR%/webapps.
  8. Start Tomcat and once up-and-running, go to the following url with your browser (if Tomcat is running on localhost with port 8080, will vary depending on your installation): http://localhost:8080/openejb/installer.
  9. Click on the ‘install’-button on the webpage, once installed, stop Tomcat.
  10. Go to the %TOMCAT_INSTALL_DIR%/config directory and edit the the openejb.xml file. Under the Resource-element named “My JMS Resource Adapter” and before the Connector-element named “My JMS Connection Factory”, add the following resources in between as defined below:

<Resource id="jdbc/tomee-maven-demo-ds" type="DataSource">
  JdbcDriver oracle.jdbc.xa.client.OracleXADataSource
  JdbcUrl jdbc:oracle:thin:@localhost:1521:XE
  UserName scott
  Password scott
  JtaManaged true
</Resource>

<Resource id="jms/tomee-maven-demo-cf" type="javax.jms.ConnectionFactory">
  ResourceAdapter My JMS Resource Adapter
</Resource>

<Resource id="jms/tomee-maven-demo-q" type="javax.jms.Queue" />

Project setup

The project is divided into 4 different modules.

  1. The jar module consists of  entity classes, the domain interface and service interface.
  2. The ejb module contains the domain EJB’s storing the entities through JPA and eclipse link and the MDB’s processing the messages stored on the queue.
  3. The war module contains the JSF 2 related stuff needed for the UI.
  4. The ear module packages all the modules for deployment on an application server.

For clarification: I wanted to put the persistence.xml and orm.xml files inside the jar-file containing the EJB’s as those are the classes using the JPA entity manager.  Unfortunately it lead to class loading issues in Weblogic as the EJB jar is packaged directly under the EAR file. The entity classes are located under the APP-INF/lib (shared lib) directory within the ear which resulted in my entity classes not being found (e.g. ClassNotFoundException).  I’ve tried adding a reference to the jar containing the entity classes with the section within the persistence.xml file, but no avail. That’s why I putted those XML-files in the jar file containing the entity classes instead of the EJB jar.

Purpose

The purpose of this demo application is to develop an asynchronous webservice named MessageService.  The webservice puts the given message on a queue through an EJB called MessageMDBSender.  There is a Message Driven Bean named MessageMDB which will process any message posted on the queue and store them in the database through an EJB called MessageEJB.  That EJB uses JPA 1.0 with Eclipse Link to store the message into the database.  If the creation of the given message is successful, the webservice will send a callback to the client who send the initial request.

I’ve also added a soapUI project to allow the testing of the deployed asynchronous webservice on Tomcat

The code that is used to develop the asynchronous webservice is based on the example posted by Biemond, except I adapted the code to make it independent of the used webservice stack.

This means that the same code will work on Metro as well as on CXF, which is necessary as weblogic uses Metro and OpenEJB uses CXF. It also works with the Oracle SOA Suite for that manner.

Beside that I also wrote a class named CXFServlet in the test package of the war project. This class is also packaged with the war when building for Tomcat as I couldn’t figure out how to make the CXF webservices work out of the box with OpenEJB.  It seems that the webservices are deployed under the root of the Tomcat installation and not under the context root of the deployed web application.  If someone could tell me how to solve this issue that would we great as it would make the demo even simpler. Another problem that I encountered with OpenEjb and Tomcat is that the resource injection through env-entries in the ejb-jar.xml isn’t working. The same ejb-jar.xml file injects the env-entries correctly on Weblogic so I suspect there might be a bug there in the 3.1.4 release of OpenEJB. The injection also fails with unit tests.

To build the project for Weblogic, just type mvn clean install. Then deploy the generated ear-file to the application server.

To build the project for Tomcat, just type mvn clean install -Ptomcat. Then copy the generated war-file to the webapps directory of the Tomcat installation directory.

You can find the complete project on https://github.com/dabla/tomee-maven-demo.

Tagged , , , , , , , , , , ,

Building and testing ADF applications with Maven, JSFUnit, Arquillian and Embedded Glassfish

Some time ago I have been playing with Java Server Faces 2.0 (e.g. JSF 2) and JSFUnit in combination with Arquillian and Embedded Glassfish as a Proof Of Concept for unit testing JSF applications.

As our applications are being developped with the Oracle ADF framework, I was wondering if it would be possible to do the same with ADF based applications.  The purpose is to build ADF applications with Maven, which is not that hard, but also test it on an Embedded Glassfish, which is quite challenging.  Once this is working, we could use Jenkins as our continuous integration platform.

The first challenge was to determine the required ADF libraries.  As those libraries are defined as a shared library in Weblogic, I had to determine which where needed to make the ADF application run on Glassfish.  To do this, I added the libraries one by one until the ClassNotFoundExceptions were gone.

Then I made a script (e.g. bat-file) which installs all those libraries in my local Maven repository.  I also made some parent POM’s so I could bundle some related libraries together in a logical way.  I know that the latest JDevelopper has an option to install those ADF libraries in Maven, but I wanted to do it manually.

Once this was done, it was only a matter of configuring Arquillian to deploy the ADF application on the Embedded Glassfish.  As I previously did a POC with JSF 2 and Arquillian, most of the configuration settings where already there for reuse. The main challenge with Arquillian, JSFUnit and Glassfish is combining the corresponding versions of the dependencies to make it all work together.  Also the ADF version that we are using runs on JSF 1.2, while Glassfish ships with version 2, so I googled how I could fix that.  Just add a sun-web.xml-file to make it work (see solution at http://stackoverflow.com/questions/2333330/jsf-1-2-app-not-working-with-glassfish-v3).

The setup of the Maven project consists of 3 sub projects, namely a Model, a ViewController and an ear project. The Model project is for holding the models of course. In this example I added a simple BC4J component.  For the moment, it still connects to a local installed Oracle database (Express Edition).  In the future, I would like to change it to a local embedded Derby database to make the tests totally isolated.  I also have to add some concrete tests to test the BC4J component completely.

The ViewController project holds my managed bean with the JSP-page, the ADF configuration and the page binding files.  The ear project speaks for itself so the main focus will be on the ViewController project, which is of course the most important one.

As I spent quite a few days on it to make it all work, I decided to share this research as I’m sure this could be useful for other ADF/JSF developers.

You can find the complete project on https://github.com/dabla/adf-maven-demo.

Note that the project holds a setup directory.  This directory contains a bat-file which adds the ADF libraries to your local Maven repository and also copies some parent POM-files for grouping purposes.

Tagged , , , , , , , , , ,
Follow

Get every new post delivered to your Inbox.