Recommendation systems, fraud detection engines, personalization platforms, and enterprise search solutions all rely on integrating data from multiple systems while preserving context and relationships.

Enterprise Knowledge Graphs (EKGs) have emerged as a foundational architecture for addressing this challenge. By modeling enterprise data as entities and relationships, EKGs enable richer semantics, improved data discoverability, and more intelligent downstream decision making.

While the conceptual benefits of knowledge graphs are well understood, scaling them to production grade digital platforms remains complex. Graph systems that perform well at small or medium scale often struggle under high ingestion rates, complex traversal queries, and strict latency requirements.

This article outlines some practical, field tested strategies for optimizing enterprise knowledge graphs for real world scalability. Rather than presenting purely theoretical models, we'll focus on architectural patterns, operational lessons, and performance insights from large scale enterprise deployments.

What We'll Cover:

• Prerequisites

• Why Scalability Becomes the Core Challenge

• Moving Beyond a Single Graph Store: Hybrid Architectures

• Partitioning for Scale: Reducing Distributed Traversal Costs

• Managing Semantic Inference Without Sacrificing Performance

• Improving Query Performance with Smarter Planning

• Observability as a First Class Requirement

• Impact on Digital Product Platforms

• Conclusion

Prerequisites

This is an architectural guide intended for data engineers, platform architects, and developers managing production-grade graph systems. To get the most out of this article, you should have the following:

Conceptual Knowledge

• A solid understanding of Enterprise Knowledge Graphs (EKGs) and the fundamental differences between RDF triple stores and Labeled Property Graphs (LPGs).

• Familiarity with distributed systems concepts, including data partitioning, semantic inference, and event-driven architectures.

Technical Background

• Experience working with real-time data integration pipelines (such as CDC, Kafka, or Pulsar).

• Familiarity with database observability, query execution planning, and general performance optimization techniques at scale.

Understanding the Enterprise Knowledge Graph (EKG)

Before exploring how to scale these systems, it's helpful to understand exactly what a knowledge graph is and how it organizes information.

At its core, a knowledge graph is a data model that represents real-world entities and the complex relationships between them. Unlike traditional relational databases that lock data into rigid, disconnected tables, knowledge graphs store data as a flexible, interconnected network.

A knowledge graph is built on three fundamental components:

• Nodes (Entities): The distinct objects, concepts, or people in your data ecosystem (for example a Customer, a Product, a Location).

• Edges (Relationships): The lines connecting the nodes that define how they interact (for example "PURCHASED," "LOCATED_IN," "MANUFACTURED_BY").

Properties: The descriptive metadata attached to nodes or edges (for example, a customer's signup date, or the price of a product).

Our Running Example: The Global Electronics Supply Chain Graph

To ground these concepts, we'll use a unified example throughout this article: an enterprise graph for a global electronics manufacturer managing product data, suppliers, and manufacturing compliance.

• Nodes (Entities): Customer (Alice), Product (NeoPhone 15), Component (MX-200 Chip), Supplier (MaxSemi), and Region (EU).

• Edges (Relationships): PURCHASED, PART_OF, SUPPLIES, and LOCATED_IN.

• Properties: The NeoPhone 15 node has properties like price: 999 and sku: "NP15-01". The PURCHASED edge has a property of timestamp: 2026-06-03.

Imagine you're building the data foundation for a retail recommendation engine. To build the graph, you move through a few distinct phases:

1. Establish ontology: First, you define the blueprint – the rules dictating what kinds of entities exist and how they are allowed to interact.

2. Define the nodes: You integrate data to generate specific entity nodes, such as a Customer node for "Alice," a Product node for "Noise-Canceling Headphones," and a Brand node for "TechAudio."

3. Map the edges: You connect these nodes based on user actions and inventory data. Alice VIEWED the Headphones. The Headphones are MANUFACTURED_BY TechAudio.

Why does this matter? Because the data is natively structured as a relationship network, the system can rapidly execute context-rich queries.

If you want to know what else Alice might buy, you don't need to write a heavy, expensive SQL query that joins millions of rows across five different tables. Instead, the graph simply "walks" the pathways you've already built. It traverses from Alice, across the VIEWED edge to the Headphones, across the MANUFACTURED_BY edge to TechAudio, and can instantly return other products connected to that same brand.

By prioritizing the relationships between data points as much as the data points themselves, EKGs provide the contextual intelligence required for modern digital products.

Why Scalability Becomes the Core Challenge

Most enterprise knowledge graph initiatives begin with a limited scope, integrating a small number of datasets, enabling semantic search, or improving reporting accuracy. Early-stage deployments often succeed using a single graph database or RDF store.

Scalability challenges emerge when EKGs become production critical infrastructure, particularly when supporting customer facing or latency-sensitive applications. At this stage, multiple pressures converge:

1. Rapid data growth as more systems and entities are integrated

2. Continuous ingestion from streaming pipelines and transactional systems

3. Increasing query complexity, including multi hop traversals

4. Strict response time requirements, often under tens of milliseconds

5. Inference overhead introduced by ontologies and reasoning engines

Simply adding hardware or scaling nodes horizontally rarely resolves these issues. Performance degradation often results from architectural mismatches between graph workloads and system design.

Moving Beyond a Single Graph Store: Hybrid Architectures

The Limits of Monolithic Graph Deployments

RDF triple stores offer strong semantic expressiveness and standards compliance but may struggle with high volume transactional updates or deep real time traversals. Conversely, labeled property graph (LPG) databases often provide efficient traversal performance but lack native semantic reasoning capabilities.

Attempting to consolidate semantic modeling, inference, operational queries, and analytics into a single system frequently results in trade offs that affect performance, cost, or maintainability.

A Pragmatic Hybrid Model

A hybrid or polyglot architecture distributes responsibilities across systems optimized for specific workloads:

1. Semantic layer (RDF / OWL): Ontology management, schema governance, reasoning workflows.

2. Operational graph layer (LPG): Real time traversals, recommendation engines, application queries.

3. Analytical stores: Aggregations, reporting, and historical analysis.

To maintain consistency between the semantic layer (RDF/OWL) and the operational graph layer (LPG), many teams implement synchronization strategies like Change Data Capture (CDC) and event driven pipelines.

In this approach, updates in one layer are captured as events and propagated to the other layer in near real time using streaming platforms such as Kafka or Pulsar. For example, updates in the operational graph can trigger semantic updates, ensuring that ontologies and relationships remain aligned.

Some systems also use dual write patterns or scheduled reconciliation jobs to detect and resolve inconsistencies. In practice, event-driven synchronization combined with periodic validation provides a balance between real time accuracy and system reliability.

This separation isolates performance critical paths while preserving semantic richness where it adds value.

In production environments, hybrid architectures consistently demonstrate improved query latency and operational flexibility compared to monolithic graph deployments, particularly for traversal-heavy workloads. Some teams have also reported latency reductions of 30–60% when separating traversal-heavy workloads into LPG layers, compared to monolithic graph deployments.

This improvement is primarily due to reduced query complexity and optimized storage for specific access patterns.

In Practice: Splitting the Supply Chain Graph

In a production-grade digital platform, a single database engine struggles to handle both semantic governance and high-speed operational queries on this data simultaneously.

Here is how the hybrid model divides the labor:

• The Semantic layer (RDF/OWL): Manages strict ontological classification and compliance rules. For example, it defines the rule: “If a Component is supplied by an entity in a country under a trade embargo, the final Product inherits a 'High Risk' compliance flag.”

• The Operational Layer (LPG): Optimized for fast, multi-hop traversals required by customer-facing apps. When Alice views the NeoPhone 15 on a mobile app, the system queries a Labeled Property Graph (like Neo4j) using a language like Cypher to instantly traverse from the product to its components for a real-time availability check:

MATCH (p:Product {id: 'NeoPhone15'})-[:HAS_COMPONENT]->(c:Component)
RETURN c.name, c.stock_level

Partitioning for Scale: Reducing Distributed Traversal Costs

As enterprise knowledge graphs outgrow single node capacity, distributed execution becomes necessary. Partitioning strategy then becomes a critical performance factor.

Why Default Partitioning Often Fails

Many graph systems use hash-based or random partitioning to distribute data evenly across nodes. While this approach balances storage, it often fragments highly connected subgraphs. Even moderately complex traversals may then require excessive cross-node communication, increasing latency and reducing throughput.

Topology-Aware Partitioning

Topology-aware partitioning colocates frequently connected entities to minimize network hops during traversal. Common approaches include:

1. Partitioning by business domain (for example, customers, products, organizations).

2. Community detection based clustering.

3. Partitioning informed by observed query patterns.

In practice, teams can achieve topology-aware partitioning by first analyzing query patterns and identifying frequently traversed relationships. Based on this analysis, related entities are co-located within the same partition to minimize cross-partition queries.

Graph processing frameworks and database tools often provide built-in algorithms for community detection, which help group highly connected nodes. Teams can also monitor query performance over time and iteratively refine partitioning strategies to align with evolving workloads.

By combining domain driven design with continuous performance monitoring, teams can incrementally optimize graph layouts without requiring major architectural changes.

In production-inspired environments, topology-aware strategies significantly reduce traversal fan out and improve both median and tail latency under concurrent load.

Though repartitioning introduces operational complexity, the performance gains justify the effort once the knowledge graph becomes central to digital product delivery.

In Practice: Partitioning by Product Domain

Let’s look at what happens when our supply chain graph scales across multiple database nodes.

If we use Default Hash Partitioning, the graph is split randomly by node IDs. Alice might end up on Machine 1, the NeoPhone 15 on Machine 2, and the MX-200 Chip on Machine 3. A query tracking whether a component shortage affects Alice's order requires a slow, expensive network hop across three separate physical servers.

Using Topology-Aware Partitioning, we can configure the cluster to use the Region or Product_Line as a partitioning key.

• Partition A (Europe Hub): Co-locates Region: EU, Product: NeoPhone 15, its internal MX-200 Chip, and local customer orders.

Result: A multi-hop traversal checking component supply chains for European customers happens entirely within local memory on a single machine, reducing query latency.

Managing Semantic Inference Without Sacrificing Performance

Semantic inference is a defining strength of EKGs but also a frequent source of scalability challenges.

The Inference Cost Problem

Applying full ontology reasoning at query time can dramatically increase computational overhead. In some systems, inference effectively multiplies graph size, increasing memory and CPU consumption. Not all inferred relationships are equally valuable for every workload.

Strategies for Selective Inference and Materialization

Scalable EKG platforms typically adopt a selective strategy:

1. Precompute and materialize frequently accessed inferences

2. Offload complex reasoning to batch or asynchronous pipelines

3. Disable low value inference paths in latency-sensitive workloads

Hierarchical classifications and role-based relationships are often materialized ahead of time, while complex rule based reasoning is reserved for offline processing. This approach stabilizes query latency and reduces peak CPU utilization in enterprise deployments.

In Practice: Materializing the Compliance Path

Recall our semantic rule: If a component has a supply risk, the final product inherits that risk.

• The Scalability Bottleneck (Query-Time Inference): Every time an enterprise dashboard loads a product catalog of 10,000 items, the engine must recursively calculate: Product -> Has Component -> Supplied By -> Supplier Country -> Embargo List. Under high concurrent load, this calculation crashes performance.

• The Optimization (Materialization): We run an asynchronous batch job or Kafka consumer that listens for supplier updates. When a supplier's status changes, it computes the inference once and writes a direct property is_high_risk: true directly onto the Product node in the operational LPG.

Now, the customer-facing application reads a simple, static property without running an expensive multi-hop recursive inference query during runtime.

Improving Query Performance with Smarter Planning

As query complexity increases, query planning becomes a decisive performance lever.

Limitations of Static Planning

Traditional graph engines often rely on static heuristics or limited statistics for execution planning. In dynamic enterprise environments where data distributions evolve, these heuristics frequently produce suboptimal execution plans, leading to unpredictable performance.

ML-Assisted Query Optimization

Machine learning techniques are increasingly being applied to query optimization, particularly for cardinality estimation. By learning from historical query execution data, ML models can predict plan costs more accurately than rule-based systems.

In controlled experiments and production pilots, ML-assisted planning has demonstrated substantial reductions in execution time for complex traversals, as well as improved consistency in response times.

While implementation requires operational maturity, this represents a promising direction for large scale graph optimization.

In Practice: Optimizing Traversal Direction

Consider this query on our data: "Find all customers who purchased a product containing the MX-200 Chip."

There are two ways the graph execution planner can execute this:

1. Plan A: Start at Component: MX-200, find the products it belongs to, and then find the customers who bought those products.

2. Plan B: Scan all Customer nodes in the database, look at their purchases, and filter for the ones containing the chip.

If the MX-200 is a rare chip used in only one niche product, Plan A is incredibly fast. If it is a generic resistor used in millions of products, Plan B or a modified hybrid plan might be more efficient.

An ML-assisted query planner analyzes the real-time cardinality (the actual count) of the PART_OF and PURCHASED relationships in your specific database instance. It prevents the graph engine from choosing a disastrously slow traversal path when data distributions shift unexpectedly.

Observability as a First Class Requirement

Scalability can't be managed without deep observability.

Beyond Infrastructure Metrics

Monitoring CPU and memory alone provides limited insight into graph-specific performance issues. Effective EKG observability includes:

1. Query level latency metrics

2. Traversal depth and fan-out tracking

3. Inference cost monitoring

4. Partition imbalance detection

Closing the Optimization Loop

By continuously analyzing these signals, teams can iteratively refine partitioning strategies, caching policies, and materialization decisions. This feedback loop improves predictability and reduces production incidents.

In practice, strong observability often distinguishes proactive optimization from reactive firefighting.

Impact on Digital Product Platforms

When applied collectively, these optimization strategies materially enhance scalability and reliability. Across enterprise deployments, teams commonly observe:

1. Reduced latency in real time workloads

2. Improved ingestion throughput under sustained load

3. Linear or near linear scaling as datasets grow

4. Greater stability during traffic spikes

These technical improvements translate directly into business outcomes: faster recommendations, more relevant search results, and increased confidence in deploying EKGs as mission critical infrastructure.

Conclusion

Enterprise knowledge graphs are no longer experimental. They're becoming the backbone of intelligent, data driven systems. As teams move toward AI-powered decision making, the role of knowledge graphs is expanding beyond storage into enabling context-aware reasoning and automation.

An optimized EKG isn't just a database – it acts as the connective tissue between data, models, and real world applications. It provides the structured context that modern AI systems, including agentic workflows and autonomous decision engines, rely on to operate effectively.

By adopting hybrid architectures, topology-aware partitioning, and intelligent query strategies, teams can build scalable and resilient graph systems that support both operational and analytical workloads.

Ultimately, organizations that invest in well-designed knowledge graph infrastructure will be better positioned to power the next generation of AI systems where retrieval, reasoning, and action are seamlessly integrated.

If you're curious about graph databases and how they compare with relational database management systems, then this beginner-friendly guide is for you.

In this article, you'll discover of the power of graphs by working with a small movie data set. It is based on the built in dataset and guide available on the Neo4j Sandbox.

Want to drive right in and have a go yourself? Please do! You’ll find instructions on how to get up and running here.

What We'll Cover in This Article

Graph databases are growing in popularity and adoption. With ever-larger amounts of data from many different sources, it's critical to be able to understand the data and see how it's all connected.

If you want to find out more about the kinds of problems graph databases help solve, and how you might spot a good application for one, here's an introductory blog post.

Some of you reading on may have heard of graph databases (GDB), some you perhaps haven’t. In this article we’re going to cover exactly what they are, and how they compare to the more traditional, Relational Database Management Systems (RDBMS) which have been the stalwart software application of the past 40+ years.

Inspired by a small movie data set used by Neo4j as a guided introduction to graph querying, we are going to look at side-by-side examples and equivalents of what a data model or query would look like in both a graph database, and a relational database.

In this article, we will:

• Introduce graph databases, briefly covering the two models that exist
• Take a conceptual look at the differences between relational and graph paradigms
• Look at the movie data set, and compare and contrast data models from a GDB and a RDBMS perspective
• Compare and contrast some queries, based on either Cypher (for GDB) or SQL
• Talk through the more interesting queries that appear in the movie example, and break out exactly what is happening

If you’d like to have a play with the example walkthrough movie data set before reading the article (or during!), you are more than welcome to do so. You can find out more here.

What is a Graph Database?

First of all, before we dive into what a graph database is, let’s define the term. Graph databases are a type of “Not only SQL” (NoSQL) data store. They are designed to store and retrieve data in a graph structure.

The storage mechanism used can vary from database to database. Some GDBs may use more traditional database constructs, such as table-based, and then have a graph API layer on top.

Others will be ‘native’ GDBs – where the whole construct of the database from storage, management and query maintains the graph structure of the data. Many of the graph databases currently available do this by treating relationships between entities as first class citizens.

Different Types of Graph Databases

There are broadly two types of GDB, Resource Descriptive Framework (RDF)/triple stores/semantic graph databases, and property graph databases.

An RDF GDB uses the concept of a triple, which is a statement composed of three elements: subject-predicate-object.

Subject will be a resource or nodes in the graph, object will be another node or literal value, and predicate represents the relationship between subject and object. There are no internal structures on the nodes or relationships, and everything is identified by a unique identifier, in the form of a URI.

The motivation behind this structure is exchanging and publishing data. To find out more about this structure, I would refer you to Jesus Barrasa’s work in this space.

A Property GDB is focussed on the concept of storing data that is close to the logical model. This in turn will be based on the questions sought of the data itself, and focuses on making that representation as efficient as possible for storage and querying.

Unlike an RDF-based graph, there are internal structures on the nodes and relationships, lending to a rich representation of data as well as associated metadata.

The following two diagrams provide a side by side comparison of sample data represented in a Property Graph Database, and as an RDF graph – both of which representing the person Tom Hanks, acting the role Jim Lovell, in the movie Apollo 13.

RDF example of Tom Hanks in Apollo 13

Property Graph example of Tom Hanks in Apollo 13

Anatomy of a Property Graph Database

For the rest of this article, we will be focussing on native property graph databases, specifically Neo4j. Let's check out the main components.

The main components of a property graph database are as follows:

• Node: also known as a vertex in graph theory – the main data element from which graphs are constructed
• Relationship: also known as an edge in graph theory – a link between two nodes. It will have direction and a type. A node without relationships is permitted, a relationship without two nodes is not permitted

Node and Relationship

• Label: Defines a node category, a node can have more than one
• Property: Enriches a node or relationship, no need for nulls!

Label, Type, and Property

Graph Databases vs Relational Databases

Relational Databases Recap

A lot of developers are familiar with the traditional relational database, where data is stored in tables within a well-defined schema.

Each row in the table is a discrete entity of data. One of these elements in the row is typically used to define its uniqueness: the primary key. It could be a unique ID or maybe something like a social security number for a person.

We then go through a process called normalization to reduce data repetition. In normalization, we’re moving references, something like an address for a person, into another table. So we get a reference from the row representing the entity to the row representing the address for that person.

If, for example, somebody changes their address, you wouldn’t want multiple versions of that person’s addresses everywhere and have to try and remember all the different instances of where that person’s addresses exist. Normalization makes sure you have one version of the data, so you can make the updates in one place.

Then when we query, we want to reconstitute this normalized data. We do what’s called a JOIN operation.

In our main entity row, we have the primary key that identifies the ID for the entity, let’s say the person. We also have what’s called a foreign key that represents a row in our address table. We join the two tables through their primary and foreign keys, and use that to look up the address in the address table. This is called a JOIN and these JOINs are done at query time and at read time.

When we’re doing a JOIN in a relational database, it’s a set comparison operation where we’re looking to see where our two sets of data overlap (in this case, the sets are the person table and the address table). At a high level, that’s how traditional relational databases work.

An example of the tables found and how they map to each other in a relational database for an insurance database

The equivalent example of the insurance database in a property graph database

How Native Graph Databases Work: Connections and Index-Free Adjacency

Let’s have a quick peek at a native graph database and how it works.

We spoke about the discrete entity in a relational database being a row within a table. In a native graph database, that row would be the equivalent of a node. It’s still a discrete entity, so we still have this element of normalization.

A node would be an entity. If we were having person nodes, we would have one node for one person. And we would have some degree of uniqueness in it, let’s say the social security number.

The key difference, however, is when we are connecting this person node to another discrete entity – for example, an address – we create a physical connection (aka relationship) between those two points.

The address would have a pointer that says, what is the outbound part of the relationship that connects to the node? We then have another pointer for the inbound part of the relationship pointing to the other node.

So, effectively, we’re collecting a set of pointers, and this is a manifestation of the physical connection between those two entities. That is the big difference.

In a relational database, you would reconstitute the data with joins on read, which means at query time, it would go off to try and figure out how things map together.

In a graph database, since we already know these two elements are connected, we don’t need to look up the mapping at query time. All we’re doing is following the stored relationships to the other nodes.

This is something we call index-free adjacency. This concept of index-free adjacency is key to understanding the performance optimizations of a native graph database compared to other database systems.

Index-free adjacency means that during a local graph traversal, following these pointers (relationships) that connect the nodes in my graph, the performance of the operation is not dependent on the overall size of the graph. It depends on the number of relationships connected to the nodes that you’re traversing.

When we talk of a JOIN being a set operation (intersection), we’re using an index in a relational database to see where those two sets overlap. This means that the performance of the JOIN operation starts to slow down as the tables get bigger.

In big O notation terms, this is something like logarithmic growth using an index — something like O(log n) and also grows exponentially with the number of JOINs in your query.

On the other hand, traversing relationships in the graph is more of linear growth based on the number of relationships in the nodes that we’re actually traversing, not the overall size of the graph.

This is the fundamental query time optimization that graph databases make that give us index-free adjacency. From a performance perspective, that is really the most important thing to think about when we think of a native graph database.

A Brief Introduction to the Movie Graph

We've spoken a fair bit about the theoretical differences between a graph and relational database. Now let's start to look at some side-by-side comparisons.

The movie graph consists of a data set consisting of actors, directors, producers, writers, reviewers and movies, along with information on how they all connect to each other.

This data set is available within Neo4j Browser, and can be easily triggered by using the :PLAY movies command. As a reminder, here's a blog to show you how to get started.

The Movies data set consists of:

• 133 Person nodes/entities
• 38 Movie nodes/entities
• 253 relationships/connections between the above entities, describing connections such as:
• Person(s) who directed a Movie
• Person(s) who acted in a Movie and role(s) played
• Person(s) who wrote a Movie
• Person(s) who produced a Movie
• Person(s) who have reviewed a Movie and score and summary given
• Person(s) who follow another Person

While it is a relatively small data set, it comprehensively describes the power of graphs.

Comparing data models

First of all, let's take a look at the data models of our respective databases. As with all data models, what they look like will ultimately depend on the types of questions you are asking. So let's assume we're going to ask the following types of questions:

• What movies did a person act in?
• What movies does a person have a connection with?
• Who are all the co-actors a person has ever worked with?

Based on these, here are the associated potential data models:

Entity Relationship data model for movie graph

Property Graph data model for movie graph

Immediately you will spot something – those IDs have gone! As we are connecting data together as soon as we know there's a connection there, we no longer need them, or those mapping tables to let us know how different rows of data connect together.

Comparing queries

Let's now move on to comparing some queries. Taking a few of the first queries from the :PLAY movies example, let's look at some side-by-side comparisons of the Cypher query, and what would the equivalent SQL query look like.

What is Cypher, I hear you ask? Cypher is a graph query language which is used to query the Neo4j graph database. There is an OpenCypher version too, which is used by a number of other vendors.

As we move through the queries, it should start to become clearer how a graph database, accompanied with a query language to help explore relationships really starts to come into its own. Let's start looking for Tom Hanks!

How to find Tom Hanks

MATCH (p:Person {name: "Tom Hanks"})
RETURN p

SELECT * FROM person 
WHERE person.name = "Tom Hanks"

How to find Tom Hanks movies

MATCH (:Person {name: “Tom Hanks”})-->(m:Movie)
RETURN m.title

SELECT movie.title FROM movie
INNER JOIN movie_person ON movie.movie_id = person_movie.movie_id
INNER JOIN person ON person_movie.person_id = person.person_id
WHERE person.name = "Tom Hanks"

How to find movies Tom Hanks has directed

MATCH (:Person {name: "Tom Hanks"})-[:DIRECTED]->(m:Movie)
RETURN m.title

SELECT movie.title FROM movie
INNER JOIN person_movie ON movie.movie_id = person_movie.movie_id
INNER JOIN person ON person_movie.person_id = person.person_id
INNER JOIN involvement ON person_movie.involve_id = involvement.involve_id
WHERE person.name = "Tom Hanks" AND involvement.title = "Director"

How to find co-actors of Tom Hanks

MATCH (:Person {name: "Tom Hanks"})-->(:Movie)<-[:ACTED_IN]-(coActor:Person)
RETURN coActor.name

WITH tom_movies AS (
    SELECT movie.movie_id FROM movie
    INNER JOIN person_movie ON movie.movie_id = person_movie.movie_id
    INNER JOIN person ON person_movie.person_id = person.person_id
    WHERE person.name = "Tom Hanks")
SELECT person.name FROM person
INNER JOIN person_movie ON tom_movies = person_movie.movie_id
INNER JOIN person ON person_movie.person_id = person.person_id
INNER JOIN involvement ON person_movie.involve_id = involvement.involve_id
WHERE involvement.title = "Actor"

More Queries with Cypher

Hopefully you’ll get the idea of the differences between Cypher and SQL queries. Perhaps you’re excited to learn more about them too! We’ll have some references further on in the blog post.

For now, let’s have a look at some of the other Cypher queries you can find in the :PLAY movies graph example, and explain what’s going on.

No movie graph would be complete without the quintessential Bacon number question, and our movie graph is no different!

Up until now, the examples we have looked at have always traversed one relationship each time. We can easily take advantage of those ‘joins on write’ to traverse many relationships to answer interesting questions.

So, back to the Kevin Bacon number. The following query will start at the Kevin Bacon person node, and then go up to 4 hops out from that start point, to bring back all connected movies and people.

MATCH (bacon:Person {name:"Kevin Bacon"})-[*1..4]-(hollywood)
RETURN DISTINCT hollywood

We can do this by using the syntax of *1..4 in the relationship part of the query pattern:

• * indicates everything
• 1..4 indicates the range - 1 says from 1 hop away, 4 says up to 4 hops away

Another graphy thing we could do on this movie data set is the shortest path between two nodes.

In this example, let’s find out the shortest path between Kevin Bacon and Meg Ryan. You will spot we’re using the * syntax again for the relationship pattern – indicating everything.

What may be new to you is the p=. You’ve seen how we use references for nodes (e.g. bacon or meg in our current query), and we can do the same for relationships.

We can also have references for the whole path (that is, all the nodes and relationships involved). The syntax we use for that is refName =, which in this example is p=.

We also use the Cypher function shortestPath() – this is a simple shortest path function that will return the first shortest path between two specified nodes. Be aware there may be another, equally short path, but this simple function will just bring back the first one encountered.

For those of you interested in other path-related functions, check out the ones available in APOC and GDS.

MATCH p=shortestPath(
(bacon:Person {name:"Kevin Bacon"})-[*]-(meg:Person {name:"Meg Ryan"}))
RETURN p

A word of warning to you all: you may see that [*] and be tempted to run your graph without the constraint of the shortestPath() function or the 1..4 range. But this may well result in something unexpected.

In our example with Kevin Bacon and Meg Ryan, even though there are only 253 relationships in this very small data set, all of the possible combinations of paths between nodes and relationships could easily run into millions of different paths between Bacon and Ryan.

When using * in your relationships as part of a query, use with extreme caution! This problem does not come up with shortest path because as a potential path that is longer than the currently identified shortest one is encountered, it is immediately dropped.

A simple recommendations query

Here are two queries that really show the power of graph databases and we can easily use the connections in our data to make some recommendations.

In our first query we’re looking for new co-actors for Tom Hanks to work with who he’s not already worked with. The query does this by:

• Firstly, finding all of the co-actors he has already worked with
• Then, finding all of the co-actors' co-actors (referred to as co-co-actors)
• Next, we want to exclude those co-co-actors who have already worked with Tom, as well as making sure the co-co-actor isn’t Tom himself
• Finally, we return the suggested co-co-actors names, and we’re going to order them but the number of co-actors that have worked with them – the more co-actors that have worked with that co-co-actor, the better the recommendation.

MATCH (tom:Person {name:"Tom Hanks"})-[:ACTED_IN]->(m)<-[:ACTED_IN]-(coActors),
(coActors)-[:ACTED_IN]->(m2)<-[:ACTED_IN]-(cocoActors)
WHERE NOT (tom)-[:ACTED_IN]->()<-[:ACTED_IN]-(cocoActors) 
    AND tom <> cocoActors
RETURN cocoActors.name AS Recommended, count(*) AS Strength 
    ORDER BY Strength DESC

Excellent, so we’ve found some potential co-co-actors. In this next query, we want to suggest Tom Cruise as a potential new co-actor for Tom Hanks to work with. But, who’s going to introduce these Toms to each other? Back to the movie graph we go.

In this query, we:

• Find the co-actors of Tom Hanks, and then find which out of those co-actors have also acted with Tom Cruise
• Then we’ll return the co-actor and the movies they were in with both Tom Hanks and Tom Cruise

MATCH (tom:Person {name:"Tom Hanks"})-[:ACTED_IN]->(m)<-[:ACTED_IN]-(coActors),
(coActors)-[:ACTED_IN]->(m2)<-[:ACTED_IN]-(cruise:Person {name:"Tom Cruise"})
RETURN tom, m, coActors, m2, cruise

Last Words

We've come to the end of our walk through of the movies database example. Hopefully those of you with a relational database background have a better idea of the similarities and differences between relational and graph databases, as well as a taste of the Cypher query language.

If you're keen to learn more about modeling and querying Neo4j, do check out the free Graph Academy.

A coming-of-age story for your database model

_Image credit: [Charlotte Parent](http://www.charlotteparent.com/CLT/Health-Development/More-Than-the-Birds-and-the-Bees-Teaching-Your-Child-About-Healthy-Sexuality/" rel="noopener" target="blank" title=")

Graph databases are a great way to store conversational data. A simple dialog tree can add depth to character interactions in a video game. A knowledge graph can extract more meaning from dialog to better understand how user intent relates to an application’s data.

In this article, I’ll show you a basic graph model for capturing chatbot interactions and how to persist them using the Apache TinkerPop framework. I’ll also show you some Gremlin queries for adding a recommendation feature to the chatbot. The source code and setup instructions for my example “Recipe Bot” are on GitHub.

Review: Recipe Bot

The Recipe Bot is a Slack Bot User that lets people request recipes based on specified ingredients or cuisines. Previously I showed you how to add support for users to request their favorite recipes, like so:

The graph version of the bot has all of the same features I discussed in my previous article on persisting metadata with JSON, but with the graph version you’ll be adding recommendations.

How it works with TinkerPop

Here is an architecture diagram of how the bot works:

_My Recipe Bot. Hey! The diagram is actually an [undirected graph](https://en.wikipedia.org/wiki/Graph_theory" rel="noopener" target="blank" title="). Who knew?

You’ll see that I’m using the Watson Conversation service. Watson Conversation lets me describe the flow of the conversation through the use of dialogs, and it helps me extract information and user intent from chat messages. You can code your own dialog tree and perform your own message parsing, or you can use tools like Watson Conversation or Botkit to help. Here is how the dialog tree for the Recipe Bot is modeled:

The Watson Conversation UI. Graphs are everywhere.

You can follow a conversation through the dialog tree similar to how you can follow vertices and edges in a graph (after all, trees are graphs too):

It’s not much of a tree, but I’m keeping it simple, y’all.

In the simplified graph above, the Recipe Bot cares only about the progression between the major entities of the bot:

1. People
2. Ingredients
3. Cuisines
4. Recipes

Data model & access pattern

As the conversation progresses, you store the following vertices and edges using the TinkerPop API:

1. Person vertex: For each person that interacts with the bot, store that person as a vertex in the graph.

{  "label": "person",  "type": "vertex",  "properties": {    "name": "U2JBLUPL2"  }}

2. Ingredient or cuisine vertex: When a person requests a specific ingredient or cuisine, you store that ingredient or cuisine — along with the list of recipes retrieved from Spoonacular — as a vertex.

{  "label": "cuisine",  "type": "vertex",  "properties": {    "name": "chinese",    "detail": "[{\"id\": 573147, \"title\": \"Kale Fried Rice\"..."  }}

3. Selects edge, person → (ingredient | cuisine): You create an edge, labelled "selects", between the person and the ingredient or cuisine (that is, “person selects cuisine”). In addition, store a "count" property on the edge and increment its value each time the user requests the same ingredient or cuisine.

{  "label": "selects",  "type": "edge",  "inV": 4152,  "outV": 4224,  "properties": {    "count": 3  }}

4. Recipe vertex: When a user requests a recipe, store the recipe as a vertex.

{  "label": "recipe",  "type": "vertex",  "properties": {    "name": "573147",    "detail": "Ok, it takes *45* minutes to make...*",    "title": "Kale Fried Rice"  }}

5. Selects edge, (ingredient | cuisine) → recipe: You create another "selects" edge between the ingredient or cuisine and the recipe (that is, “cuisine selects recipe”). In addition, store a "count" property on the edge and increment it each time the ingredient or cuisine selects the same recipe.

{  "label": "selects",  "type": "edge",  "inV": 4320,  "outV": 4152,  "properties": {    "count": 22  }}

6. Selects edge, person → recipe: You create yet another "selects" edge directly between the person and the recipe (that is, “person selects recipe”). Store a "count" property on the edge and increment it each time a person requests the same recipe.

{  "label": "selects",  "type": "edge",  "inV": 4320,  "outV": 4224,  "properties": {    "count": 4  }}

7. Has edge, recipe → (ingredient | cuisine): Finally, create an edge, labelled "has", between the recipe and the ingredient or cuisine (that is, “recipe has cuisine”). This relationship allows you to find all the ingredients and cuisines that a recipe uses. There is no count field on this edge.

{  "label": "has",  "type": "edge",  "inV": 4152,  "outV": 4320}

The graph for a single user looks something like this:

This graph has it going on. It’s a [(weakly) connected graph](https://en.wikipedia.org/wiki/Connectivity%28graph_theory%29#Definitions_of_components.2C_cuts_and_connectivity" rel="noopener" target="blank" title="). There are all kinds of graphs.

So far, by using a graph database, you get the following benefits:

1. Reduce third-party API calls by caching entities.
2. Provide a more personal experience for users by harnessing metadata on their interactions.

A “more personal experience” for Recipe Bot means allowing users to request their favorite recipes. To find a user’s favorite recipes, you use the Gremlin graph traversal language. The following Gremlin query will give you a user’s top-five favorite recipes, sorted by count:

Adding recommendations

Since you track every user interaction with the bot as a graph, you can find popular ingredients, cuisines, or recipes requested by all users. You can use Gremlin to find popular recipes based on an ingredient or cuisine. Here’s how it works:

Let’s say, a user is looking for recipes that use onions:

You can find popular recipes that use onions by issuing the following query. (I’ll unpack it further below—don’t worry!):

This query says, “Give me anyone, excluding the calling user, who has requested recipes more than once that have onions.” It breaks down like so:

1. Start with "onions":

g.V().hasLabel("ingredient").has("name","onions")

2. Get the recipes that have "onions". This API call uses the "has" edge coming from the recipe vertex into the ingredient vertex. Using .in() skips the edge and only returns the recipe vertex. (You don’t need any properties from the edge object, so there’s no reason to return it here.)

.in("has")

3. Get the users that have requested these recipes more than once. This call uses the "selects" edge coming from the person to the recipe:

.inE().has("count",gt(1)).order().by("count", decr)

4. Get the users, excluding the current user:

.outV().hasLabel("person").has("name",neq("CURRENT_USER"))

5. Get the full path:

.path()

This call returns an array of matching paths that looks like this:

ingredient ← recipe ← edge ← person

You can access these recommended recipes at index 1.

When you return this recipe list to the user, the app puts the recommended recipes at the top and highlights the number of users who have previously used which recipe:

What’s Next?

Try a deployment for yourself. The project’s README has step-by-step instructions for completing your first deployment on IBM Bluemix. There’s also a Java port of the example app.

If you’re already using a dialog tree in your applications and want to use a graph database to persist metadata on interactions, I hope the source code in the repo above gives you some ideas on delivering more personalized experiences to your users.

And if you’ve enjoyed this article, please hit the ol’ ♥ so other Medium users might find it and dig it too. Happy coding!