Hi all,
I would like to discuss this idea for TinkerPop 4.
# Proposal: Provider-focused interfaces for efficient data retrieval
When building Gremlin providers - whether connecting to remote databases,
in‐memory stores, or custom backends - there’s a recurring challenge:
TinkerPop’s core traversal engine asks for full element payloads, while
many providers only need subsets of data (selected properties or specific
edge directions) to satisfy a step. This mismatch leads to:
- Unnecessary data transfer or object materialization
- Wasted CPU cycles filtering out unused fields
- Hard-to-maintain provider code packed with ad hoc “if this step needs X,
fetch Y” logic
Introducing three lightweight interfaces - `PropertiesProcessor`,
`AdjacentProcessor`, and `ElementReader` - offers a clean extension point
that lives in provider code without bloating the TinkerPop core.
---
## Proposed Interfaces
```java
public interface PropertiesProcessor {
enum PropertiesRequirement {
NONE,
CUSTOM,
ALL,
}
// second value is properties
Pair<PropertiesRequirement,String> getPropertiesRequirement();
}
public interface AdjacentProcessor {
enum AdjacentRequirement {
NONE,
IN,
OUT,
BOTH
}
// second value is edge labels. Null if all labels are required.
Pair<AdjacentRequirement, String> getAdjacentRequirement();
}
public interface ElementReader {
void setElementRequirement(Pair<PropertiesRequirement, String>
propertiesRequirement,
Pair<AdjacentRequirement, String>
adjacentRequirement);
}
```
---
## Why This Helps Providers
- **Performance Gains**
By knowing exactly which properties and adjacent edges are required,
providers can:
- Send targeted queries to the backend (e.g., read only needed
columns/properties)
- Avoid loading full element metadata into memory
- Reduce network bandwidth for remote stores
- **Clear Separation of Concerns**
Core TinkerPop remains focused on traversal semantics. Providers
implement these interfaces to translate greedy element requests into
optimized backend calls.
- **Consistent Extension Point**
Instead of scattering `if (step instanceof X)` checks across provider
code, every step that implements `PropertiesProcessor` or
`AdjacentProcessor` exposes its requirements via the same API.
- **Easier Maintenance & Evolution**
Future provider enhancements - like supporting richer filters or
push-down queries - build upon these interfaces. There’s no need to touch
core TinkerPop logic, only provider‐side readers.
---
## Impact on TinkerPop Core
- No new dependencies in the core module
- Default behavior: if a step doesn’t implement these interfaces, providers
fall back to fetching full elements
---
## Conclusion
Adding `PropertiesProcessor`, `AdjacentProcessor`, and `ElementReader` as
provider‐focused interfaces creates a lightweight, uniform way for Gremlin
providers to express data needs. This yields measurable performance
improvements, cleaner code, and a more maintainable separation between
TinkerPop’s traversal engine and its many backend implementations.
Thank you,
Valentyn
