We use the following definition for an event. The event is based on business entities and the operations executed on those entities. We use the following definition:
BusinessObject The name of the business entity, i.e. Invoice, Customer
Operation Created, Updated, Deleted
TrackingId Tracking ID so that flows can be followed
SourceSystem The system that published the event
TargetSystem The target system to which the event must be sent explicitly. This is an optional field
Identifiers This is used to add meta data, i.e. invoice number, location, order number
Message The message. This contains the content of the business entity
Master data entities
This works fine for master data entities, because you want to use events to indicate that entities are created, updated or deleted. We use this to decouple the MDM solution from the interested applications. New applications are easily added by making a SOA composite subscriber for that application.
In this case the solution falls short, because the context is missing. For example when an Invoice is received from a customer, you want to send the event InvoiceReceivedFromCustomer. In our case we use the Created operation on Invoice but this does not say much. Within the meta data we had to add more context information.
- Think about the names you use for Events, i.e. Invoice_Create is too vague and will need more metadata to filter on the correct event
- Think about the data you put in the message
Master data entities can be especially good candidates to publish
Data will change
The content of entities will change, i.e. fields are added. After this change you want to synchronize the interested applications with this new information. This is especially needed for master data. So you are decoupled on the publishing- and subscribing applications, but still strongly coupled to the data. You need to synchronize the new dataset to the applications (batch). There are several options:
Use ETL to synchronize the data
This can be a good option in case it is lots of data. Extra tooling and data transformation is needed, which you tried to avoid. The source and target systems are again coupled.
Publish all data and make use of the Pub-Sub implementation
This works fine in case there are less entities. You don’t need extra ETL tooling but re-use the publish-subscribe mechanism. We use this and also use the targetSystem field within the event, when we know it is only relevant to a particular application(s).
- Think about data reconciliation in case data fields are changed
- Try to avoid ETL because this couples the source and target systems again
What I see often in integration projects, is that each interested application a new integration is built particularly for that application. For example a new supplier wants to retrieve Invoice messages from the company. However it wants to retrieve the invoice in its own format (for example csv) and based on its own protocol (for example sftp). The application, that generates the invoices, just exports yet another file. What I propose is to let the publishing application generate Invoice events. It should have no knowledge about the interesting applications. It just has to do the jobs. Then for each supplier make a Subscribe composite that filters the Invoices targeted for that supplier and map the Invoice to the suppliers format and protocol. This is where the Subscriber composite filters come in.