Hi Valentyn, I can see how your proposal would be beneficial for providers. Could you provide an example of step implementation of those interfaces? The example(s) would be helpful to communicate how you envision the proposed interfaces to be used by providers.
Thanks, Andrea From: Valentyn Kahamlyk <[email protected]> Date: Tuesday, July 22, 2025 at 1:58 PM To: [email protected] <[email protected]> Subject: [DISCUSS] Proposal for provider-focused interfaces for efficient data retrieval Hi all, I would like to discuss this idea for TinkerPop 4. # Proposal: Provider-focused interfaces for efficient data retrieval When building Gremlin providers - whether connecting to remote databases, in‐memory stores, or custom backends - there’s a recurring challenge: TinkerPop’s core traversal engine asks for full element payloads, while many providers only need subsets of data (selected properties or specific edge directions) to satisfy a step. This mismatch leads to: - Unnecessary data transfer or object materialization - Wasted CPU cycles filtering out unused fields - Hard-to-maintain provider code packed with ad hoc “if this step needs X, fetch Y” logic Introducing three lightweight interfaces - `PropertiesProcessor`, `AdjacentProcessor`, and `ElementReader` - offers a clean extension point that lives in provider code without bloating the TinkerPop core. --- ## Proposed Interfaces ```java public interface PropertiesProcessor { enum PropertiesRequirement { NONE, CUSTOM, ALL, } // second value is properties Pair<PropertiesRequirement,String> getPropertiesRequirement(); } public interface AdjacentProcessor { enum AdjacentRequirement { NONE, IN, OUT, BOTH } // second value is edge labels. Null if all labels are required. Pair<AdjacentRequirement, String> getAdjacentRequirement(); } public interface ElementReader { void setElementRequirement(Pair<PropertiesRequirement, String> propertiesRequirement, Pair<AdjacentRequirement, String> adjacentRequirement); } ``` --- ## Why This Helps Providers - **Performance Gains** By knowing exactly which properties and adjacent edges are required, providers can: - Send targeted queries to the backend (e.g., read only needed columns/properties) - Avoid loading full element metadata into memory - Reduce network bandwidth for remote stores - **Clear Separation of Concerns** Core TinkerPop remains focused on traversal semantics. Providers implement these interfaces to translate greedy element requests into optimized backend calls. - **Consistent Extension Point** Instead of scattering `if (step instanceof X)` checks across provider code, every step that implements `PropertiesProcessor` or `AdjacentProcessor` exposes its requirements via the same API. - **Easier Maintenance & Evolution** Future provider enhancements - like supporting richer filters or push-down queries - build upon these interfaces. There’s no need to touch core TinkerPop logic, only provider‐side readers. --- ## Impact on TinkerPop Core - No new dependencies in the core module - Default behavior: if a step doesn’t implement these interfaces, providers fall back to fetching full elements --- ## Conclusion Adding `PropertiesProcessor`, `AdjacentProcessor`, and `ElementReader` as provider‐focused interfaces creates a lightweight, uniform way for Gremlin providers to express data needs. This yields measurable performance improvements, cleaner code, and a more maintainable separation between TinkerPop’s traversal engine and its many backend implementations. Thank you, Valentyn Warning: The sender of this message could not be validated and may not be the actual sender.
