What is Camel

Why an Integration Framework

Integration is Critical for Business

Framework does the Heavy Lifting

Lets you Focus on the Problem

Prevents Re-inventing the Wheel

History of Camel

Started in March 2007

Founded by

• James Strachan
• Rob Davies
• Hiram Chirino
• Guillaume Nodet

First Release in June 2007

Spawned from Apache ServiceMix and Apache ActiveMQ

Simple Programming

Minimal Configuration

DSL Based Programming

6 DSL's and Counting

• Java DSL
• Spring XML
• Blueprint XML
• Rest DSL
• Groovy DSL
• Scala DSL
• Kotlin DSL in the works

Java DSL

          from("file:src/data?noop=true")
  .choice()
    .when(xpath("/person/city = 'London'"))
      .to("file:target/messages/uk")
    .otherwise()
      .to("file:target/messages/others");
        

Spring XML

          <route>
  <from uri="direct:a"/>
  <choice>
    <when>
      <xpath>$foo = 'bar'</xpath>
      <to uri="direct:b"/>
    </when>
    <when>
      <xpath>$foo = 'cheese'</xpath>
      <to uri="direct:c"/>
    </when>
    <otherwise>
      <to uri="direct:d"/>
    </otherwise>
  </choice>
</route>
        

Rest DSL

                
rest("/say")
  .get("/hello").to("direct:hello")
  .get("/bye").consumes("application/json").to("direct:bye")



                
              

                <rest path="/say">
  <get uri="/hello">
    <to uri="direct:hello"/>
  </get>
  <get uri="/bye" consumes="application/json">
    <to uri="direct:bye"/>
  </get>
</rest>
              

Components

191 Components and counting

Component for pretty much anything

Most commonley used components

• CXF
• CXFRS
• JMS
• ActiveMQ
• AMQP
• Infinispan
• CDI
• Drools

Lots of Components

More Components

So Many Components

Hundreds of Components

Error Handling

Try...Catch...Finally

Exception Clause in Java DSL

Error Handlers

• Default Error Handler
• Dead Letter Channel
• Logging Error Handler
• No Error Handler
• Transaction Error Handler

Support for Exponential Backoff of Retries

Error Handling Code

      from("direct:start")
  .doTry()
    .process(new ProcessorFail())
    .to("mock:result")
  .doCatch(IOException.class, IllegalStateException.class)
    .to("mock:catch")
  .doFinally()
    .to("mock:finally")
  .end();
    

      onException(ValidationException.class)
  .to("activemq:validationFailed");

onException(ShipOrderException.class)
  .to("activemq:shipFailed");

from("seda:order").to("bean:processOrder");
    

      errorHandler(deadLetterChannel("jms:queue:dead")
  .useOriginalMessage()
  .mamimumRedeliveries(5)
  .redeliverDelay(5000));

errorHandler(defaultErrorHandler()
  .allowRedeliveryWhileStopping(false)
  .maximumRedeliveries(20)
  .redeliveryDelay(1000)
  .retryAttemptedLogLevel(LoggingLevel.INFO));
    

Redelivery Delay Patterns

Allows for custom backoff of retries.

Syntax: limit:delay;limit2:delay2;limit3:delay3;...;limitN:delayN

If delayPattern=5:1000;10:5000;20:20000 then we get

• Redelivery attempt number 1..4 = 0 sec (as the first group starts with 5)
• Redelivery attempt number 5..9 = 1 sec (the first group)
• Redelivery attempt number 10..19 = 5 sec (the second group)
• Redelivery attempt number 20.. = 20 sec (the last group)

Security

Four Broad Categories of Security Offered

• Route Security
• Payload Security
• Endpoint Security
• Configuration Security

Route Security

Policy driven security for routes or route segments

• Shiro Security
• Spring Security

Payload Security

Encryption and decryption services for secure paylods

• XMLSecurity DataFormat (XML Encryption support)
• XML Security component (XML Signature support)
• Crypto DataFormat (Encryption + PGP support)
• Crypto component (Signature support)

Endpoint Security

Some components can be secured, but not all

• Jetty - HTTP Basic & SSL
• CXF - HTTP Basic & WS-Security
• Spring Web Services - HTTP Basic & WS-Security
• Netty - SSL
• MINA - SSL
• Cometd - SSL
• JMS - JAAS and SSL for client <--> broker communication

Configuration Security

Externalize configuration properties

• Properties
• Jasypt - Encrypted values
• JSSE Utility - Configure SSL/TLS

Testable

Native test frameworks

Supports unit and integration Testing

Built in mock & stub support

Advanced testing with NotifyBuilder and AdviceWith

Works with 3rd party test frameworks

• PaxExam
• Arquillian
• JUnit
• TestNG

Available Test Harnesses

Example Camel Test

            public class FilterTest extends CamelTestSupport {
  @EndpointInject(uri = "mock:result")
  protected MockEndpoint resultEndpoint;
  @Produce(uri = "direct:start")
  protected ProducerTemplate template;
  @Test
  public void testSendMatchingMessage() throws Exception {
    String expectedBody = "matched";
    resultEndpoint.expectedBodiesReceived(expectedBody);
    template.sendBodyAndHeader(expectedBody, "foo", "bar");
    resultEndpoint.assertIsSatisfied();
  }
  @Override
  protected RouteBuilder createRouteBuilder() {
    return new RouteBuilder() {
      public void configure() {
        from("direct:start").filter(header("foo").isEqualTo("bar")).to("mock:result");
      }
    };
  }
}
          

Tooling

Several tools exist for working with and monitoring Camel

• Camel Maven Plugin
• JBoss Tools for Camel
• Talend Open Studio
• Hawtio Camel Plugin

JBoss Tools for Camel

JBoss Tools for Camel

Enterprise Integration Patterns

Similar to Design Patterns

Focused specifically on Integration

Written by Gregor Hohpe & Bobby Wolf

65 Documented Patterns

Camel supports 53 of them

Enterprise Integration Patterns

EIP to Camel

                  
Endpoint newOrder = endpoint("activemq:queue:newOrder");
Predicate isWidget = xpath("/order/product ='widget'");
Endpoint widget = endpoint("activemq:queue:widget");
Endpoint gadget = endpoint("activemq:queue:gadget");

from(newOrder)
  .choice()
    .when(isWidget)
      .to(widget)
    .otherwise()
      .to(gadget)
  .end();
                  
                

Camel and Red Hat

Included in JBoss Fuse

Included in JBoss Fuse Service Works

• Run as part of SwitchYard
• Run outside of SwitchYard on EAP Container

Supported on EAP 6.1.1+

• Requires JBoss Fuse & JBoss EAP Subscription
• Must use Camel libraries from JBoss Fuse

JBoss Fuse

JBoss Fuse Service Works

Fuse or Service Works

Use Fuse if...

• Dont use Java or JEE
• Dont use EAP
• Use OSGi or Karaf
• Governance is not needed
• No SCA requirement

Use Fuse Service Works if...

• ​Use JEE
• Want to use JEE in Services
• Currently use EAP
• Need Governance
• Need SCA or already use SwitchYard

Camel Considerations

For Fuse...

• Can use Blueprint/Spring/Java DSL for Routes
• Most configuration is done via XML
• Testing relies on PaxExam and Built in Camel Test Frameworks

For Fuse Service Works w/ SwitchYard...

• SwitchYard limits amount of Components you can use
• JEE Stack well Supported
• No XML Configuration
• Can make use of Arquillian

For Camel on EAP...

• Can use CDI or Spring
• JEE Stack not currently well supported
• Most Components use Spring for configuration
• Can make use of Arquillian

Camel on EAP

Use Camel without SwitchYard or Karaf

Supported on EAP 6.1.1 and up with Camel 2.12 Red Hat Libraries

Two ways to use Libraries

• Packaged inside Application
• Deployed as EAP Module

Camel Started with 3 methods

• Spring ContextLoaderListener
• ServletListener Component
• CDI Component

Starting with Spring

Requires Spring

Best if already using Spring

Camel uses Spring's Bean Registry

Started using Spring ContextLoaderListener

Configuration very similar to Fuse with Spring or Blueprint

Spring Example

Example web.xml

            
<web-app ...>
  <display-name>soap-contract-first</display-name>

  <!-- Config Location -->
  <context-param>
    <param-name>contextConfigLocation</param-name>
    <param-value>classpath:META-INF/spring/*.xml</param-value>
  </context-param>

  <!-- Spring Listener Servlet -->
  <listener>
    <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
  </listener>
</web-app>
            
          

Spring Example

Example camel-route.xml

            
<beans ...>

  <import resource="classpath:META-INF/spring/camel-cxf.xml" />
  <camelContext xmlns="http://camel.apache.org/schema/spring">
    <route>
      <from uri="cxf:bean:orderEndpoint" />
      <log message=">> Received SOAP Endpoint: ${body}"></log>
      <setBody>
        <simple>DONE</simple>
      </setBody>
    </route>
  </camelContext>

</beans>
            
          

Starting with Spring Demo

https://github.com/bsaunder/camel/tree/master/eap_6/soap-contract-first

Starting without Spring

Does not Require Spring

Best if using JEE, but do not need to access Camel with JEE Stack

Camel uses a Simple Map based Bean Registry (Can also use JNDI)

Started using CamelServletContextListener

Configuration XML Based

• Route Builders defined in web.xml

ServletListener Configuration

Example web.xml

            
<web-app ...>
  <display-name>servlet_listener</display-name>

  <!-- Set CamelContext name -->
  <context-param>
    <param-name>name</param-name>
    <param-value>myCamelContext</param-value>
  </context-param>
  <context-param>
    <param-name>routeBuilder-JavaRoute</param-name>
    <param-value>net.bryansaunders.camel.servlet_listener.JavaRoute</param-value>
  </context-param>
  <context-param>
    <param-name>routeBuilder-XmlRoute</param-name>
    <param-value>classpath:camel-routes.xml</param-value>
  </context-param>
  <listener>
    <listener-class>org.apache.camel.component.servletlistener.SimpleCamelServletContextListener</listener-class>
  </listener>

</web-app>
            
          

ServletListener Component Demo

https://github.com/bsaunder/camel/tree/master/eap_6/servlet_listener

Bootstrapping with CDI

Does not Require Spring or use Servlets

Best if using JEE, Especially CDI

Camel uses CDI Bean Manager

Started using an @Startup @Singleton Annotated Bean

All Configuration done in Java, No XML

Camel Bootstrap Class

Must be a Singleton, @Singleton

Must run at Startup. @Startup

Inject an instance of CdiCamelContext

• Adds a CDI Bean Registry that allows lookup of CDI Beans when Instantiated

@PostConstruct method should Configure/Start Context and Add Routes

@PreDestroy should Stop Camel Context

Example Bootstrap Class

            
@Singleton
@Startup
public class Bootstrap {

  @Inject
  private CdiCamelContext camelCtx;
  @Inject
  private SomeRouteBuilder someRoute;

  @PostConstruct
  public void init() throws Exception {
    this.camelCtx.addRoutes(someRoute);
    this.camelCtx.setName("my-camel-context");
    this.camelCtx.start();
  }

  @PreDestroy
  public void stop() throws Exception {
    this.camelCtx.stop();
  }
}
            
          

Using the XML Based DSL

Spring and Blueprint XML routes can also be used

Routes loaded from XML file in Bootstrap Class

            
InputStream is = this.getClass().getClassLoader().getResourceAsStream("camel-routes.xml");
if (is != null) {
  RoutesDefinition routes = this.camelCtx.loadRoutesDefinition(is);
  this.camelCtx.addRouteDefinitions(routes.getRoutes());
}
            
          

CDI Component Demo

https://github.com/bsaunder/camel/tree/master/eap_6/cdi-soap-consume

Configuration with Camel CDI

No Spring Config or Context

With Camel CDI Components are Configured in Java

Configured on the Camel Context

All Components can be Configured

Configuring Components

Configred in your Bootstrap Class

Must be Configured Prior to Start of Camel Context

Lookup Existing Components with getComponent()

• Lookup via Registered Name - jms, activemq, etc...
• Most Core Components already Registered on Context
• Lazily initialize Components that arent Registered

Add New Components with addComponent()

Sample Configuration

            
@Resource(mappedName = "java:/ConnectionFactory")
private ConnectionFactory jmsConnFactory;

...

JmsConfiguration jmsConfig = new JmsConfiguration(jmsConnFactory);
jmsConfig.setConcurrentConsumers(10);
jmsConfig.setMaxConcurrentConsumers(10);

Component component = this.camelCtx.getComponent("jms");
if (component != null) {
  JmsComponent jmsComponent = (JmsComponent) component;
  jmsComponent.setConfiguration(jmsConfig);
} else {
  this.camelCtx.addComponent("jms", new JmsComponent(jmsConfig));
}
            
          

JMS Configuration Demo

https://github.com/bsaunder/camel/tree/master/eap_6/cdi-jms

Camel & JBoss A-MQ

Camel has no JBoss A-MQ Component

Several ways to Connect to A-MQ from Camel

• JMS Component
• ActiveMQ Component
• Bean using JMS API

Camel JMS vs ActiveMQ

JMS component

• Based on Spring JMS
• Uses Spring JmsTemplate for Sending
• Uses Spring MessageListenerContainer for Consuming

ActiveMQ Component

• Based on JMS Component
• Optimized for ActiveMQ
• Supports ActiveMQ Specific Features such as Destination Options

Attachments over JMS

Not Supported by Camel or any of its JMS based Components

Two Solutions

• Move Attachment into Message Body with a Custom Object
• Use the Claim Check EIP

Claim Check EIP

Replaces Message Data with a Unique ID for Future Retrieval

Data stored into Persistent Data Store

Data Retrieved later using Unique ID

Claim Check EIP Example

            
              from("direct:start").to("bean:checkLuggage").to("jms:queue:SomeQueue");
              from("jms:queue:SomeQueue").to("bean:dataEnricher").log("Do Stuff");
            
          

            
              public static final class CheckLuggageBean {
                public void checkLuggage(Exchange exchange, @Body String body, @XPath("/order/@custId") String custId) {
                  dataStore.put(custId, body);
                  exchange.getIn().setHeader("claimCheck", custId);
                  exchange.getIn().setBody(null);
                }
              }

              public static final class DataEnricherBean {
                public void addDataBackIn(Exchange exchange, @Header("claimCheck") String claimCheck) {
                  exchange.getIn().setBody(dataStore.get(claimCheck));
                  dataStore.remove(claimCheck);
                  exchange.getIn().removeHeader("claimCheck");
                }
              }
            
          

Web Services with Camel CDI

Web Services are common place in Integration

Camel has Several Web Service Related Components

• CXF
• CXFRS
• Spring Web Services
• Rest
• Spark-Rest

Some of the Problems

Spring Web Services require Spring

Rest and Spark-Rest Components not Available

• Part of Camel 2.14

Only Leaves CXF and CXFRS

• Dependent on Spring for Configuration
• Needs Servlet to Start

Using Camel CXF with Camel CDI

Configured via Camel Context

Endpoints created Programmatically

Requires Http Jetty Transport to Publish Service

            
CxfEndpoint orderEndpoint = new CxfEndpoint();
orderEndpoint.setAddress("http://localhost:9595/order");
orderEndpoint.setServiceClass("net.bryansaunders.camel.OrderEndpoint");
orderEndpoint.setWsdlURL("wsdl/order.wsdl");
orderEndpoint.setCamelContext(this.camelCtx);
camelCtx.addEndpoint("cxf:bean:orderEndpoint", orderEndpoint);
            
          

Alternate Solution

Use the Web Services Subsystem

Deploy Web Services using JAX-WS

Use Producer Template to Call Camel from Implementation

Use a Direct Route

Route is more Flexible, Can be Called without SOAP

Easier to Configure more Advanced Web Services

JAX-WS Demo

https://github.com/bsaunder/camel/tree/master/eap_6/cdi-jaxws-publish

Camel & SwitchYard Test Support

Camel and SwitchYard both include Test Frameworks

Mocks created with Camel Test cannot replace SwitchYard Endpoints

• Intercept Messages and Route to Mockable Endpoint

SwitchYard expects Route to Start with SwitchYard Endpoint

• Define two Routes
• First Route has SwitchYard Endpoint and Forwards to Second Route Second Route starts with a Mockable Endpoint (Usually Direct)

Camel & SwitchYard Test Support

            
              public class SomeTestClass extends CamelTestSupport {
                private static final String MESSAGE = "...";
                @Test
                public void someTest() throws Exception {
                  MockEndpoint endpoint = getMockEndpoint("mock:switchyard:WebOutService");
                  endpoint.expectedBodiesReceived(MESSAGE);
                  template.sendBody("direct:WebInService", MESSAGE);
                  endpoint.assertIsSatisfied();
                }
                @Override
                protected RouteBuilder createRouteBuilder() throws Exception {
                  return new WebInServiceRoute() {
                    @Override
                    public void configure() throws Exception {
                      interceptSendToEndpoint("switchyard://WebOutService")
                      .skipSendToOriginalEndpoint().to(
                      "mock:switchyard:WebOutService");
                      super.configure();
                    }
                  };
                }
              }
            
          

            
              from("switchyard://WebInService").to("direct:WebInService");
              from("direct:WebInService").log("Doing something very clever").to("switchyard://WebInCanonicalSystem");
            
          

Testing with Pax Exam

In-Container Testing Framework for OSGi, Java EE, and CDI

Similar to Arquillian

Uses a Test Driver and a Test Container

• Test Driver launches the OSGi framework and the system under test
• Probe Builds Bundles from Test Cases and Injects into the Container

JUnit and TestNG Drivers

Drivers are Annotation Based

Pax Exam

Supports All Major OSGi Frameworks

• Equinox, Felix, Knoplerfish, Karaf

Multiple Strategies for Restarting and Reusing the Running OSGi Framework for each Test

Two Types of Containers

• Native runs in the same VM as the Test Driver
• Forked runs in a separate VM from the Test Driver

Pax Exam Karaf Container

Eases Integration Testing with Pax Exam and Karaf

Provides an Actual Karaf Container for Testing

Supports any Karaf Based Distribution (Fuse, Service Mix, Geronimo)

Maintained as Official Pax Exam modules

Adds Karaf Specific Support

Container Configuration

Controls the Host Container

Determines the Set of Bundles and Features provisioned to the Container

Builds and Configures the Container Environment

Multiple Methods of Specifying Configuration Options

• One or more Methods Annotated with @Configuration that return Option[]

Probe

Artifact added to the Container for Testing

Created on the Fly by Pax TinyBundles

Contains the Current Test Classes and All Classes/Resources under the Same Root

Can be Configured inside the Test Class if needed

Writing Testable Routes

Anything that needs to Change during a Test should be Externalized

• All Endpoints
• Configuration Values
• Header Names
• Etc...

Integrations with External Resources should be Mock able

• Database Connections
• Messaging Brokers
• Etc...

Generating Dependency File

Generate using the Maven Plugin

            
<plugin>
  <groupId>org.apache.servicemix.tooling</groupId>
  <artifactId>depends-maven-plugin</artifactId>
  <version>1.2</version>
  <executions>
    <execution>
      <id>generate-depends-file</id>
      <goals>
        <goal>generate-depends-file</goal>
      </goals>
    </execution>
  </executions>
</plugin>
            
          

Should use the versionAsInProject() method in Container Configuration

Configuring the Container

Container Configuration is Done with the @Configuration Annotation

Method should Return Option[]

Used for

• Specifying the Distribution
• Setting Distribution Properties
  • Log Level, Unpack Directory, Start Method, Etc…
• Loading Features / Bundles
• Adding/Modifying Distribution Environment
  • Property Files, Etc…

Example Configuration

            
@Configuration
public static Option[] configure() throws Exception {
  return new Option[] {
    karafDistributionConfiguration()
      .frameworkUrl(
        maven().groupId("org.apache.karaf").artifactId("apache-karaf").type("zip")
          .versionAsInProject()).useDeployFolder(false).karafVersion("3.0.0")
      .unpackDirectory(new File("target/paxexam/unpack/")),
    logLevel(LogLevel.WARN),
    features(
      maven().groupId("org.apache.camel.karaf").artifactId("apache-camel").type("xml")
        .classifier("features").versionAsInProject(), "camel-blueprint", "camel-jms",
      "camel-jpa", "camel-mvel", "camel-jdbc", "camel-cxf", "camel-test"),
    KarafDistributionOption.editConfigurationFilePut("etc/org.ops4j.pax.url.mvn.cfg",
      "org.ops4j.pax.url.mvn.proxySupport", "true"),
    keepRuntimeFolder(),
    KarafDistributionOption.replaceConfigurationFile("etc/com.walmart.mqm.gateway.routes.cfg", new File(
      "src/test/resources/com.walmart.mqm.gateway.routes.cfg")),
    mavenBundle().groupId("net.bryansaunders").artifactId("routeBundle").versionAsInProject() };
}
            
          

Modifying the Probe

Modified Using the @ProbeBuilder Annotation

Uses the Following Method Signature

              
@ProbeBuilder
public TestProbeBuilder probeConfiguration(TestProbeBuilder probe) {
  // Do Things
}
              
            

Method should return the probe Parameter after Modification

Sets Probe Specific Configurations via Headers

Dynamically Imports all Packages by Default

Using Provisional Packages

Provisional Packages are by Default NOT Imported

Probe must be Modified to Import Them

Add the Following Line to your ProbeBuilder Method

              
probe.setHeader(Constants.DYNAMICIMPORT_PACKAGE, "*;status=provisional");
              
            

Getting the Camel Context

There will potentially be Multiple Camel Contexts running in the Container

Need to get the Correct Context for Our Tests

Best Done in doPreSetup() Method that can be Overridden from CamelTestSupport

Camel Context should be Looked Up by it’s name in the BundleContext

            
@Override
protected void doPreSetup() throws Exception {
  camelContext = PaxExamTestUtil.getOsgiService(CamelContext.class,
    "(camel.context.name=" + CAMEL_CONTEXT_NAME + ")", 10000, bundleContext);
  assertNotNull(camelContext);
}
            
          

Sending Messages

2 Steps

• Create/Start Producer Template
• Send Message using Template

            
// Set Headers To Be Sent
final Map<String, Object> headerMap = new HashMap<String, Object>();
headerMap.put("WM_MSG_ID", null);
headerMap.put("WM_HO_WMQ_QUEUE", null);

// Send the Message Body
ProducerTemplate template = camelContext.createProducerTemplate();
template.start();

template.send("direct:gateway_in", new Processor() {
  public void process(Exchange exchange) {
    Message in = exchange.getIn();
    in.setBody("Hello from Camel");
    in.setHeaders(headerMap);
  }
});
            
          

Using Mock Queues

Get the Endpoint from the Camel Context and Cast to a MockEndpoint

              
MockEndpoint mockNoHomeOfficeDlq = (MockEndpoint)
CamelContext.getEndpoint("mock:gateway_noHomeOfficeDlq");
              
            

Set Mock Endpoint’s Expectations

              
mockNoHomeOfficeDlq.expectedMessageCount(1);
mockNoHomeOfficeDlq.expectedBodiesReceived("Hello from Camel");
              
            

Send Messages to Mock Endpoint

Assert Mock Endpoint is Satisfied

              
                mockNoHomeOfficeDlq.assertIsSatisfied(2500);
              
            

Pax Exam on Karaf Demo

https://github.com/bsaunder/camel/tree/master/fuse/pax-exam

What About Arquillian?

Arquillian has OSGi support

Works like standard Arquillian

Supports Multiple Containers

• JBoss (Embedded)
• Felix (Embedded)
• Equinox (Embedded)
• Karak (Embedded/Managed/Remote)

Not as Feature Rich as Pax Exam

• Evolving Quickly