Hi Dominik, This is a very cool feature and I already tried it. I really like how fast it is and it definitely improves the development experience and reduces the amount of (debug) dashboard sinks in a pipeline ;)
While trying, I have encountered a problem. I had a pipeline with a filter, which should only let an event through from time to time. Unfortunately this was not displayed in the preview. When I started this with a data lake sink, the correct values arrived. I suspect the reason for this behavior is because we poll the values every two seconds in the preview component. Do you have any ideas how to fix this? Because as a user you then think that the pipeline does not work as expected. Philipp > On 21. May 2021, at 12:24, Dominik Riemer <[email protected]> wrote: > > Hi all, > > > > after talking to some first-time users I discovered that our live dashboard > is often used as some kind of “debugging feature” to better see if the > created pipeline performs as expected. This leads to quite a confusing and > time-consuming pipeline development approach (creating a pipeline, appending > dashboard sinks everywhere, going to the dashboard, checking the output, > going back to the pipeline editor to change the pipeline and so on). > > So I quickly implemented a “preview feature” that can be enabled in the > pipeline editor and then shows the output for each pipeline element in form > of a table below the element (data streams & data processors and limited to 1 > event/sec to avoid web browser performance issues). > > In the background, a pipeline is started and stopped and we reuse the > pipeline element live data fetcher that was already implemented for data > streams. > > > > It would be great if you could check if the feature is working as expected or > if any changes are needed or critical bugs occur, as the implementation of > the feature seemed to be a bit too easy and I’ve expected it to be much more > complicated 😉 > > > > Dominik > > > > >
