Collaboration and Rational Tooling

Greg Hodgkinson, Methodology Pratice Leader

Running effective software development projects requires a certain level of tool support. For most organizations, the tools they have come from multiple vendors, and may also include a mix of both licensed and open-source tooling. As tools tend to be acquired to meet point problems, it can be easy to lose sight of the big picture – how your team and tools fit together to deliver the business solutions needed. Integration (SOA) projects bring with them a new set of challenges which may require tooling of a type that you don’t currently have. Do you find yourself asking any of the following?

• How do I drive my team’s work in a way that allows me to track progress?
• How do I figure out what work and changes went into a build of the code created 6 months ago?
• How do I keep my distributed team aware of each other’s actions and progress?
• How do I ensure that my best practices are being adhered to?

These are questions we hear on a regular basis from our customers. Having tested and used various tooling, we have had great success with Rational Team Concert which is highly effective for team collaboration. Below are some slides I've put together which illustrate what I'm referring to...

Greg Hodgkinson is the Methodology Practice Leader at Prolifics. He has worked in software architecture since 1996, initially in the field of component-based development (CBD), then seamlessly on to service-oriented architecture (SOA). His extended area of expertise is the software development process, and he assists Prolifics and IBM customers in adopting agile software development processes and SOA methods. Complementing this is his expertise in software development environment architecture. He is still very much a practitioner, and has been responsible for service architectures for a number of FTSE 100 companies. He presents on agile SOA process and methods, has co-authored a Redbook on SOA solutions, and regularly writes for DeveloperWorks.

Think Fast! Rational AppScan using a SaaS Model

A quick thought for you...

With all the buzz around SaaS these days there's a cool application of Rational AppScan for Web Application Security that may be of interest. You can now purchase Rational AppScan using a SaaS model. It basically is an outsourced version that is hosted and managed by security experts at IBM. You buy a subscription service so there is no infrastructure cost and setup time. Could be a good way to get started...

Dynamically Loading Java Modules

Mike Hastie, Solutions Director


Did you ever feel there should be a way to modularize your Java modules and dynamically load and unload them as needed? In fact, move away from the whole classpath headache altogether? Well, you’re not alone. The OSGi Alliance (http://www.osgi.org/Main/HomePage) - Open Services Gateway initiative, a name that is now obsolete - is a non-profit corporation founded in March 1999 with the mission to develop a standard Java-based service platform that can be remotely managed.

In case you think OSGi is new, there are OSGi frameworks in many of the systems we use, examples include:

• Eclipse – Integrated Development Environment Plug-ins are OSGi modules


• Eclipse Equinox – Server Framework


• WebSphere Application Server v6.1


• Lotus Expeditor


• Jonas v5


• JBoss is replacing JMX with OSGi


• Spring Dynamics is an OSGi implementation
  

The OSGi framework started out as a Dynamic Module and Class Loader framework for cell phones and mobile devices, but is now widely accepted as one of the leading standards. In fact, it is detailed in the JSR291 specification. OSGi is a mature standard and is in its 4th release and the 5th release is in the works. Sun is working on the JSR277 specification as a Java Modularity standard planned for Java 7, but it is thought to include compatibility with JSR291. More information is available at http://en.wikipedia.org/wiki/OSGi.

So if you have a server side development project where you have to reload classes and modules without a server restart, then take a little closer into OSGi or maybe just build a Java application for your cell phone.


















Mike Hastie is an experienced solutions architect and implementation manager with a strong background in business driven and improvement focused IT solutions. He has over 20 years of IT experience covering project management, enterprise architecture, IT governance, SDLC methodologies, and design/programming in a client/server and Web-based context. Prior to joining Prolifics, Mike was a co-founder and Director of Promenix, a successful systems integrator focused on IBM software implementations. Mike also has significant large-scale systems implementation experience using SAP ERP, data warehouses, and portals during employment with Deloitte Consulting and Ernst & Young where he specialized in messaging and integration technologies using the WebSphere brand family.

The Curious Case of Web Services Migration - Part II

Vladimir Serebryany, Senior Consultant

As I mentioned in the previous blog entry, the final goal was to develop a Maven-powered environment in which the deployable unit - in our case the EAR file - would be generated by a set of scripts starting from checking out versioned code from PVCS. I would lie if I tell you that I liked Maven at first sight. My initial feelings were anger and frustration as Maven-enabled RAD essentially diminished RAD to be just a fancy code editor - and nothing more. Gone were round-trip interactive development, and my productivity as a developer really suffered. Only by the end of the project I found out how to keep RAD effective and fully engaged and still be able to build using Maven. At the very end of the project I started to really appreciate the effectiveness of Maven as a build tool especially when it comes to building deployable units for different environments in a very uniform and reliable way. Its ability to manage versioned dependencies is just outstanding given the relative simplicity with which this is achieved.

Anyway, for regular Dynamic Web Projects the task would be trivial - Maven already has a plug-in to generate all deployable artifacts. But for Web projects with Web services there was nothing I could use because there were a number of generated artifacts which simply did not fit into Maven’s rigid default directory structure. I faced a task of developing a new Maven plug-in just for the tasks on hand (as long as I am mentioning this, you may guess that plug-in was successfully implemented, but read below to find out at what cost).

To assess the scope of the effort involved, let me just list the tasks accomplished.
Given: set of RAD projects fresh from PVCS each with Maven pom.xml. One specialized project had all WSDL and XSD files. Parent folder had pom.xml with our custom Maven Web services plug-in properties and configurations. Note that only scripts and libraries available with WAS 6.1 run-time are used for code generation. No part of RAD is used for Maven plug-in (in fact our customer uses AIX to run build scripts).

Below is the overview of custom plug-in functionality:

• For each WSDL in parent pom.xml run WSDL2Java script, create temporary folder structure with all Java and XML configuration files.
• For each generated webservices.xml go in and replace generated placeholder with actual servlet class name (this is because WSDL2Java script just does not do that by design).
• Combine all webservices.xml files together and create single webservices.xml to be put in the target project.
• Collect all mapping xml files and put them together in the target project.
• Collect all servlets names and classes names for each WSDL and put them as servlet/servlet-mapping entries in the Web Deployment Descriptor in the target project.
• Copy WSDL and XSD files into the target project WEB-INF/wsdl folder.
• Copy all generated Java sources and put them together with existing source code.
• Pass resulting project to the standard Maven Web project plug-in to be compiled and built.

At the end the Prolifics team had accomplished everything the client asked us to do. and we left the site with application up and running in production with no problems. The only thing they asked with amazement was: "How did you do that, guys?"

Vladimir Serebryany is a Senior Consultant at Prolifics with in-depth knowledge and broad hands-on experience with the J2EE environment as well as expertise in EJBs, Servlets, JSP and JSF. Excelling at migrations, Vladimir has over 9 years of experience with a wide range of complementary skills including WebSphere, WebSphere MQ, WebSphere Business Integration Message Broker, UNIX, C/C++, Java, HTML/ASP, JavaScript, and Visual Basic. He has served as team leader/senior developer roles in large, complex projects and configurations with clients in the financial, insurance and telecommunication industries - among others.

The Curious Case of Web Services Migration - Part I

Vladimir Serebryany, Senior Consultant

During one of my recent BEA WebLogic to IBM WebSphere migration assignments at a major insurance company, I encountered an interesting problem.

The client had a large number of Web services which were running on BEA WebLogic and consumed by a .NET front-end. In the course of migration, our team had to not only migrate code to WebSphere but also help the customer refactor the source code repository and create a set of Maven scripts to provide a fully scripted build/deploy process for all environments - from unit test to production.

We started with the typical set of issues:

• WSDL files and corresponding XSD files had relative namespaces in them - and the spec strongly recommends absolute ones
• Source code was stored in typical WebLogic structures and deployable components were created by running four (!) different Ant scripts
• Code and WSDL files were stored in PVCS and the revision history had to be preserved
• WSDL2Java wizard was invoked every time scripts ran so binding Java code would be generated on the fly and not stored in PVCS - wise decision but binding implementation classes with client's code in them had to be preserved from being overwritten.

The first issue was easy to deal with – a couple sed/awk scripts took care of the namespaces. We gave WSDL files back to client; they validated them against their .NET environment and found no problems with new namespaces.

Item #2 was also relatively easy to solve - we manually rearranged code into a Rational Application Developer (RAD) structure and that was it. RAD is really good at that sort of thing. While doing that we also extracted PVCS archives as files from the central repository and rearranged these archives in the file system according to the new directory structure. We sent back these directories to the client’s PVCS team, and they imported these archives back into a new PVCS project. That way all the previous generations of source code have been preserved. We now were able to work with PVCS and track a file’s revision history right back to Noah's Ark.

Now we had to deal with WSDL to Java files generation. There were three sorts of files: custom code with business logic, binding code and classes generated from data types definitions in XSD files, and binding implementation classes. We did not have to worry about the first two types: the first set was permanent code which we shifted to another utility’s Java project for simplicity of maintenance. Generated Java was what it was - generated code which the wizard took care of, only the binding implementation classes was something we had to take care of. On one hand all the methods stubs are generated by the WSDL2Java wizard - on the other hand these methods already contained the customer's code. If we allowed the wizard to run "freely," this code would be overwritten.

Here we needed a trick and we found one. As it is well-known, RAD may designate one or more folders in the project to be the Source folders - meaning it will compile all Java code in these and only these folders into binary form. It happens that built-in RAD wizards are very sensitive to the order in which Source folders are listed in "Order and Export" tab of "Java Build Path" configuration page. If the WSDL2Java wizard for bottom-down Web services generation is invoked either from the pop-up menu or from the wsgen.xml script, it always places all generated files into the folder listed first in "Order and Export" tab. That "observation" provided us with opportunity to solve the problem. We would define two Source folders: “src/main/java” and “src/main/java2.” “src/main/java” would be listed first in "Order and Export" tab, “src/main/java2” would be the second. The single "target/bin" folder was designed for binary files. We placed all existing binding implementation files into “src/main/java2” folder, and we put WSDL and XSD files into a separate project. (Note that by using “src/main/java” folder structure we were shooting to use Maven.)

From that point on, if a WSDL definition changed, we would clean “src/main/java” folder and run the Web services wizard/script against new WSDL/XSD files. Newly generated binding files would go into “src/main/java” folder. As long as binding implementation files were also on the classpath, WSDL2Java wizard was smart enough to not regenerate *BindingImpl.java files, but rather pick them up from “src/main/java2” folder. IBM provided ws_gen Ant task, script and sample properties files were customized to run the whole task at any time for all 20 WSDL's we had in the service definition (see http://publib.boulder.ibm.com/infocenter/radhelp/v7r0m0/topic/com.ibm.etools.webservice.was.creation.core.doc/ref/rtdwsajava.html). We didn't include “src/main/java” folder into PVCS and never had to check-in transient generated code.

The remaining issue to solve was to reproduce in RAD the style in which WebLogic's WSDL2Java script generated the code. By tweaking WSDL2Java options on Window->Preferences->Web services->Websphere->JAX-RPC Code Generation->WSDL2Java tab in RAD, we matched the style of generated code so that RAD-generated code was almost drop-in replacement for WebLogic-generated code. The problem was solved - but only for development phase so far.

Coming in Part II: “My initial feelings were anger and frustration as Maven-enabled RAD essentially diminished RAD to be just a fancy code editor - and nothing more. Gone were round-trip interactive development, and my productivity as a developer really suffered. Only by the end of the project I found out how to keep RAD effective and fully engaged and still be able to build using Maven…”

Vladimir Serebryany is a Senior Consultant at Prolifics with in-depth knowledge and broad hands-on experience with the J2EE environment as well as expertise in EJBs, Servlets, JSP and JSF. Excelling at migrations, Vladimir has over 9 years of experience with a wide range of complementary skills including WebSphere, WebSphere MQ, WebSphere Business Integration Message Broker, UNIX, C/C++, Java, HTML/ASP, JavaScript, and Visual Basic. He has served as team leader/senior developer roles in large, complex projects and configurations with clients in the financial, insurance and telecommunication industries - among others.

Database Design and the Demands of Operational Business Intelligence

Anant Gupta, SOA Practice Director

When hearing the term Business Intelligence the first thought that springs to mind is that of reporting and trend analysis based on historical data. So what exactly is Operational Business Intelligence (BI)? It almost seems to be an oxymoron. In one sentence, it is the mechanism of enabling everyone, and not just the strategy team as in the case of BI, to make decisions that improve the top-line, bottom-line or enhance customer satisfaction. These decisions are not made quarterly or yearly but on a transactional basis. A very simple example of operational intelligence is your GPS system telling you to take a different route to work because of a bad traffic condition on your regular route. To make this more interesting, let’s assume that your system not only understands the traffic conditions, but also gas prices, your schedule, tolls, etc., and needs to present you the most effective option based on all of these factors. To do this, it needs to perform a lot of analysis in real-time. Similarly, systems need to look at the product / promotion info and customer purchase history / status and determine cross-sell, up-sell opportunities.

So far we have been talking about transactional databases or data warehouses / data marts, one for OLTP and the other designed strictly for post-facto reporting and analysis. The new trend will be a middle path where we will be designing data marts that will have the capability to be queried and performing analysis in real time. There will also be a trend that will emerge to update the operational data stores with updates from the data warehouse. Essentially, data warehouses will be able to continuously analyze data, recognize and send events that operations will subscribe to. For example, I might be interested in subscribing to an event if merchandise returns for one of my product lines exceed 5% of normal returns. So, databases will need to start understanding events and perform continuous analysis to determine the occurrence of any of those events.

All of this imposes challenges to the database design as both the depth of analysis and performance will have to be delivered simultaneously.

Anant Gupta was recently named the SOA Practice Director at Prolifics after serving as a Senior Business Integration and J2EE architect Anant has with extensive experience in IBM's SOA software portfolio and specializes in delivering business integration and business process management solutions. He has worked for major clients in the banking, insurance, telecommunications and technology industries.

Web Application Security Considerations

Andy Blank, Security Practice Solution Director

Should you consider doing anything about Web Application Security? Here’s a quick self assessment test for all the many application developers and infrastructure specialists out there. Answer the following questions as honestly as possible.

While building customer web applications (Portals, UI front ends to business processes, SOAInfrastructure, message flows, Web services, etc.), I think of security:

• First, foremost in priority, and continuously throughout!
• As one step in my development process.
• As a separate set of tools (identity management, access management, etc.) handled by a security team.
• Security? Isn’t that the job of the imposing looking person in the elevator lobby?
  

I would guess that the applications I develop/deploy have significant security holes:

• Never! I am a gift to modern development perfection!
• Half the time.
• Three quarters of the time.
• I really have no way of knowing.
  

Unless you honestly answered ‘a’ to both questions, you should take a hard look hard look at Web Application Security -- including coding practices, vulnerability testing, managing system access, and system configurations. 75% of all current internet based attacks are made against the application layer. In addition, security companies such as Symantec surmise that up to 80% of existing web applications have at least one significant exploitable vulnerability.

Since joining Prolifics in 1994, Andrew Blank has held key positions such as Senior Technical Support Engineer, Manager of Training Services, Senior Consultant, and Migrations Practice Manager. Currently, as one of Prolifics’ Solution Directors and as leader of the Security Practice, Andrew takes part in the design, development and delivery of Prolifics’ projects for such clients as Marsh & McLennan, MetLife and UPS. His expertise in J2EE architectures, portal solutions, IT security, and systems monitoring is integral to the company’s strategic planning for adoption and use of new product technologies.