stefaniesinner opened a new issue, #4428:
URL: https://github.com/apache/streampipes/issues/4428

   ### Body
   
   ## Problem
   Currently, the adapters use lazy loading to browse the node tree which keeps 
the initial load fast. However, **OPC UA information models** often contain 
deep, nested structures with 10,000+ nodes. Finding a specific node by 
expanding the tree level by level can take minutes.
   
   ## Solution approach
   I suggest adding a **search bar** above the node tree in the adapter 
configuration. 
   
   To keep the load on the OPC UA server minimal, the idea is:
   
   ### 1. Browse the tree once
   When the adapter connects, the backend walks through the node tree once and 
returns a list of all nodes with only lightweight metadata, such as `nodeId` or 
`browseName`.
   
   To minimize server load, the initial browse:
   - batches `BrowseNext` calls
   - caches `ReferenceDescriptions` in the backend
   - uses continuation points efficiently
   
   Heavier attributes such as `dataType` or `description` stay lazy-loaded and 
are only fetched when the user clicks a node.
   
   ### 2. Build a search index in the browser
   The frontend builds a **tokenized index** from the list inside a **Web 
Worker**, so indexing does not block the main thread. The index is built once 
after the initial browse and supports:
   - full-text search over `BrowseName`, `DisplayName`, `NodeId`, 
`TypeDefinition`, and custom metadata fields
   - prefix matching, substring matching, and exact matches
   
   All searches run in the browser. There are no further round-trips to the OPC 
UA server.
   
   ### 3. Filter results
   In addition to free-text search, users can narrow results by:
   - node class (Object, Variable, Method, DataType)
   - namespaces
   - TypeDefinition (e.g., `MachineryItemState`, `AnalogItemType`, `FolderType`)
   
   ### 4. Show matches directly in the existing tree
   - Clicking a search result expands the path to that node and highlights it 
in the existing tree
   - Non-matching branches stay collapsed but visible, so the user keeps the 
hierarchical context
   - "Collapse all" button resets the tree
   - "Expand path" button opens only the path to a selected node without 
expanding unrelated branches
   
   ### 5. Keep the UI responsive
   - Search input is debounced (150-200 ms) so the index is queried only after 
the user pauses typing
   - Only the first 200 matches are rendered in the DOM with a "Load more" 
button for the rest
   - Remaining matches stay in the index and are not discarded
   
   ## Performance Targets
   | Metric | Target |
   |---|---|
   | Search across 10k nodes | < 50 ms |
   | Navigation expand latency | < 100 ms |
   | UI interaction | > 50 FPS |
   
   ## Are you willing to submit a PR?
   Yes, I am willing to submit a PR!
   
   ### StreamPipes Committer
   
   I acknowledge that I am a maintainer/committer of the Apache StreamPipes 
project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to