Integration with Graph Database Neo4j
There are many databases on the market, but most of them are categorized into three data models relational, document, and graph. Data models and characteristics of the application to store and retrieve data are the factors to choose databases, and we do not compare which one is better. Very clear that the document model is the best for use cases where data is self-contained documents and no relationships of many-to-one or many-to-many between documents. The graph model is the opposite, and it targets to use cases where the data relationship is more complicated.
Neo4j is the most popular graph database now. In this blog, we use it to implement two functionalities as the diagram below.
- Find related products: like most of the e-commerce websites, whenever a user views a product, he will see a list of related products that tempt him to buy more. The logic to find related products is different from each website, and here we are going to find products that were sold together with this product in orders that bought by any customers.
- Create sales order: when a user makes an order, the e-commerce website sends it to the back office to process, then the sales order may be forwarded to another app to handle. In this case, we will send the sales order to the Neo4j database for storage and doing predictable tasks such as “find the related products.”
We use Advantco Neo4j Adapter to connect with Neo4j database. It supports native Cypher statements, which can store and retrieve data directly from the database, and it also uses their owner schema on top of Neo4j API to store and retrieve data. We will explore these two features throughout this implementation.
Find related products
We find products that were ordered together with this product in the past orders to recommend to customers.
The interface provides a REST API endpoint for the e-commerce website to look up the related products based on the viewing product. We won’t detail the REST channel configuration here but will express the request body later.
This interface we use the native Cypher statement feature of the Advantco Neo4j adapter, so no need for any message mapping. All we need is a Cypher query statement to get the data we want. The consumer of the REST API will send out the Cypher query as the request payload, and the adapter sends it to the Neo4j database and responds the result back to the consumer.
The simple channel configuration to work with Neoj4 database
There is no specific operation such as storage or retrieval in the channel configuration, only the connectivity information. Notice that we use the message protocol of “Native Neo4j Cypher Statement”, so the Cypher query that sent from the consumer will be wrapped to the appropriate Neo4j API, and the adapter takes care of this step.
There are other authentication methods from the basic one, such as Kerberos, LDAP, and Custom.
We can use any REST adapter or HTTP adapter to expose an endpoint API with the HTTP POST method as below.
https://server:port/AdvantcoRESTAdapter/RESTServlet/neo4j/api/v1/cypherqueryThe Cypher query to get related products. We can use any as long as it returns the expected result based on our data model. In this case, we are viewing a product with ID 4 on the app, and the app looks for products that related to the product with ID 4.
match(p:Product {productID:"4"})<-[:ORDERS]-(o:Order), (o)-[:ORDERS]->(rp:Product) where rp.categoryID = p.categoryID return rpThe sample of using the exposed API with CURL. It simulates the consumer which is the e-commer website. We just past the Cypher query as the request body.
curl --location --request POST 'server:port/AdvantcoRESTAdapter/RESTServlet/neo4j/api/v1/cypherquery' \
--header 'Content-Type: application/json' \
--header 'Content-Type: text/plain' \
--data-raw 'match(p:Product {productID:"4"})<-[:ORDERS]-(o:Order), (o)-[:ORDERS]->(pp:Product) where pp.categoryID = p.categoryID return pp'
Create Sales order
We do not focus on how the e-commerce website sends the order to the back-office, but we will go through the implementation that the back-office sends the sales order to the Neo4j database.
The previous interface of finding related products we don’t use message mapping by using the native Cypher query, we can see how fast to implement an interface with that way. But within this interface, we create a sales order from an IDOC, so we need to transform an IDOC structure to a structure that could accept by the Neo4j database, it is where we use the Advantco Schema on top of Neo4j API.
Let say we have a data model as below. We will use the Advantco Workbench to generate the schema needed for the mapping.
Base on the data model, we will create a new Order node and two edges (data relationships) of PURCHASED and ORDERS.
If you are from the data model perspective, you may ask why we need a schema for a schema-free database. It’s because we need the source and target data structure to do the data transformation in the middleware. In case we need more dynamic on data structure, we can refer to the way we implement “Find related products” to use native Cypher statements.
The sample mapping from IDOC to the schema
The payload looks like after the mapping
<?xml version="1.0" encoding="UTF-8"?>
<multiRequest>
<request>
<operation>MERGE</operation>
<Order>
<ID>orderID</ID>
<LABEL>Order</LABEL>
<propertiesToReturn>id</propertiesToReturn>
<properties>
<customerID type="string">match (c:Customer {customerID: "LETSS"}) return c.customerID</customerID>
<orderID type="string">0003365889</orderID>
<orderDate type="string">2020-03-24</orderDate>
</properties>
</Order>
<PURCHASED>
<START_ID>match (c:Customer {customerID: "LETSS"}) return c</START_ID>
<END_ID>0003365889</END_ID>
<TYPE>PURCHASED</TYPE>
</PURCHASED>
<ORDERS>
<START_ID>0003365889</START_ID>
<END_ID>match (p:Product) where p.productID in ["1"] return p</END_ID>
<TYPE>ORDERS</TYPE>
<properties>
<unitPrice type="double">1719.0010</unitPrice>
<productID type="string">1</productID>
<quantity type="int">10</quantity>
</properties>
</ORDERS>
<ORDERS>
<START_ID>0003365889</START_ID>
<END_ID>match (p:Product) where p.productID in ["2"] return p</END_ID>
<TYPE>ORDERS</TYPE>
<properties>
<unitPrice type="double">020</unitPrice>
<productID type="string">2</productID>
<quantity type="int">20</quantity>
</properties>
</ORDERS>
<ORDERS>
<START_ID>0003365889</START_ID>
<END_ID>match (p:Product) where p.productID in ["3"] return p</END_ID>
<TYPE>ORDERS</TYPE>
<properties>
<unitPrice type="double">0.30</unitPrice>
<productID type="string">3</productID>
<quantity type="int">30</quantity>
</properties>
</ORDERS>
<ORDERS>
<START_ID>0003365889</START_ID>
<END_ID>match (p:Product) where p.productID in ["4"] return p</END_ID>
<TYPE>ORDERS</TYPE>
<properties>
<unitPrice type="double">0.40</unitPrice>
<productID type="string">4</productID>
<quantity type="int">40</quantity>
</properties>
</ORDERS>
</request>
</multiRequest>
The channel configuration is not much different from the previous except the message protocols of Neo4j. The payload itself tells what to do (CREATE/UPDATE/DELETE/QUERY) on which nodes and edges, no need other specify on the channel configuration unless we want to customize different settings.
Note that we are using XML payload here, but the adapter also supports other payload types such as CSV and JSON.
Conclusion
It’s fast and easy to fulfill the requirement of integration with the Neo4j database on the SAP platform with the Advantco adapter.
From the integration perspective, it’s good to know the Cypher statement to either using native Cypher or using Advantco Schema to manipulate data through the adapter.
References
Cypher statement
https://neo4j.com/docs/cypher-manual/current/introduction/
Free Online Neo4j sandbox to play with graph database
https://neo4j.com/sandbox/
Advantco Neo4j adapter
https://advantco.com/product/adapter/neo4j