Reading compressed files with Oracle Service Bus
February 27, 2012
Originally Posted by James Goddard
reasonably common B2B interface convention is to compress the contents of exchanged files, to reduce the impact on network traffic and archiving requirements (see Figure below). Although Oracle Middleware does not support such interfaces “out of the box,” it is reasonably straight-forward to piece together a simple adapter using existing tools.
This recipe will guide you through a sample implementation of a proxy service which polls a directory for XML files compressed using the gzip utility.
Gzip is a data compression software application, most commonly encountered as the version implemented by the GNU Project. Gzip only natively supports the compression of one file at a time (although of course that file may itself be the combination of several smaller files), and has a simple format consisting of compressed binary content between a standard header and footer.
J2SE includes a standard library of compression/decompression utilities in the java.util.zip package, which, among other things, includes functionality for working with the gzip data compression algorithm.
Oracle Enterprise Pack for Eclipse
Although it is possible to develop for the Oracle Service Bus using only the Web Console interface, for a fully featured IDE (including support for version control, source code editing plug-ins, lower latency and independence of development environments) it is generally preferable to use the OSB Workshop perspective of the OEPE.
This recipe assumes the use of an existing OSB Configuration project within the OSB Workshop for development so ensure that you have installed and familiarised yourself with it prior to beginning.
Prior to beginning this recipe, you will want to prepare some test data, consisting of XML files compressed using gzip.
How to do it...
Writing the Java method
Before jumping into creating anything for OSB, you’ll first need a java library which handles the extraction.
- Start up Eclipse and switch to the Java perspective.
- Right-click in the Project Explorer area and select New > Java Project.
- Name the project GzExtractor and then select Next.
- Select the Libraries tab and click Add External JARs....
- Select com.bea.core.xml.xmlbeans_188.8.131.52.jar from the modules subdirectory of your Oracle middleware installation and click Open.
- Click Finish in the Add External JARs dialog, and then Finish again in the New Java Project dialog box.
- Right click on the new project and select New->Class.
- Set the package to com.rubiconred and the class name to GzExtractor. Leave everything else as default and then click Finish.
- Replace the contents of the new file GzExtractor.java with the code below.
Incorporating the Jar File
The following steps explain how the java method created above can be incorporated into an OSB project.
- Switch to the Oracle Service Bus perspective.
- If you don’t have one already, create a new OSB Configuration Project and a new OSB project within it. Name the project GzFileAdapter.
- Right click on the GzExtractor Java project and select Export->Export....
- Select Java->JAR file and then click Next.
- Provide the export destination as GzFileAdapter/GzExtractor.jar and then click Finish.
- Right click on the GzFileAdapter OSB project and select Import->Import....
- Select File System in the General category and then click Next.
- Select the com.bea.core.xml.xmlbeans_184.108.40.206.jar XMLBeans library from the modules subdirectory of your Oracle middleware installation, as used earlier in the Java project, and then click Finish.
- Right click on GzExtractor.jar in the Project Explorer and select Oracle Service Bus -> Modify Jar Dependencies from the context menu.
- Select the XMLBeans library, then click Add, then OK.
Creating the Proxy Service
The next step is to create the file adapter, as a binary file-based message proxy service, from the Oracle Service Bus IDE.
- Right-click on the GzFileAdapter OSB project in the Project Explorer and select New -> Proxy Service.
- Name it GzFileAdapter and click Finish.
- In the General tab of the Proxy Service editor, select Message Service as the Service Type.
- In the Messaging tab, select Binary as the Request Message Type. Leave the Response Message Type as None.
- In the Transport tab, select file as the Protocol and provide a directory name where you would like to poll for files from as the Endpoint URI.
- In the File Transport tab, customise the adapter to suit. In this example, we’ll select a file mask of *.gz as well as configuring an archive Post Read Action and specifying all the necessary Directory file destinations.
Defining the message flow
The next step is to incorporate a call to the GzExtractor java library into the Proxy Service Message Flow.
- Start by dragging a new Pipeline Pair from the Design Palette into the Message Flow editor.
- Add a new Stage to the Request Pipeline and name it Extract GZIP.
- Drag an Assign Action into the new Stage.
- In the Properties tab for the Assign action, provide the Expression as $body/ctx:binary-content and theVariable as gzBinaryContent.
- Drag a new Java Callout Action from the Design Palette to just under the Assign action.
- In the Properties tab for the new Java Callout, select Browse next to the Method field.
- Select GzExtractor/com.rubiconred.GzExtractor.processFile.
- Select the Expression for the only parameter and specify $gzBinaryContent.
- Specify the Result Value variable as payload.
Testing the implementation
To test the new file adapter, you can place a GZIP’d file in the input directory and confirm the contents are read successfully following these steps:
- Create a new Reporting Stage below the GZIP Extract Stage and drag a new Log Action in.
- Set the Log Expression as $payload and the Annotation as “GzFileAdapter message payload”.
- Change the Severity to Error so that it is guaranteed to appear on your weblogic’s console output.
- Deploy the service to your server.
- Copy a GZIP’d text file to the input directory you specified earlier.
- Observe that the contents are written to the console.