[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870125#comment-16870125 ] Tomoko Uchida commented on LUCENE-8793: --- Sorry for the delay, I modified the patch a little (the final patch is here: [^LUCENE-8793.patch]) and committed to the master and 8.x branch. Thanks [~johtani]! > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Assignee: Tomoko Uchida >Priority: Minor > Attachments: LUCENE-8793-2.patch, LUCENE-8793.patch, > LUCENE-8793.patch, LUCENE-8793.patch, Screen Shot 2019-05-06 at 10.00.57.png, > Screen Shot 2019-05-07 at 1.40.47.png, Screenshot from 2019-05-06 > 13-45-40.png, Screenshot from 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870122#comment-16870122 ] ASF subversion and git services commented on LUCENE-8793: - Commit 9dabdda54acca53bccb66a3fdc96e21a8e4ee86f in lucene-solr's branch refs/heads/branch_8x from Tomoko Uchida [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9dabdda ] LUCENE-8793: Luke enhanced UI for CustomAnalyzer: show detailed analysis steps. Co-authored-by: Jun Ohtani Co-authored-by: Tomoko Uchida > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Assignee: Tomoko Uchida >Priority: Minor > Attachments: LUCENE-8793-2.patch, LUCENE-8793.patch, > LUCENE-8793.patch, Screen Shot 2019-05-06 at 10.00.57.png, Screen Shot > 2019-05-07 at 1.40.47.png, Screenshot from 2019-05-06 13-45-40.png, > Screenshot from 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16870120#comment-16870120 ] ASF subversion and git services commented on LUCENE-8793: - Commit 8e81f47ca69251cb88393626645525b57219474c in lucene-solr's branch refs/heads/master from Tomoko Uchida [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8e81f47 ] LUCENE-8793: Luke enhanced UI for CustomAnalyzer: show detailed analysis steps. Co-authored-by: Jun Ohtani Co-authored-by: Tomoko Uchida > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Assignee: Tomoko Uchida >Priority: Minor > Attachments: LUCENE-8793-2.patch, LUCENE-8793.patch, > LUCENE-8793.patch, Screen Shot 2019-05-06 at 10.00.57.png, Screen Shot > 2019-05-07 at 1.40.47.png, Screenshot from 2019-05-06 13-45-40.png, > Screenshot from 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843392#comment-16843392 ] Tomoko Uchida commented on LUCENE-8793: --- Hi Ohtani san, thanks for updating the patch, I looked through the code. I'd like to do some small layout adjustment and refactoring (e.g. move table model classes from the Operator interfaces to the concrete Provider classes) on my local branch before committing it to the master, but as a whole it looks fine to me. [~thetaphi]: Would you please look over the newly added method {{o.a.l.luke.models.analysis.AnalysisImpl#analyzeStepByStep(String)}}? There are some tricks partially copied from Solr's {{o.a.s.handler.AnalysisRequestHandlerBase}} to debug token streams, and I'm not confident in pushing the changes as is. > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Assignee: Tomoko Uchida >Priority: Minor > Attachments: LUCENE-8793-2.patch, LUCENE-8793.patch, > LUCENE-8793.patch, Screen Shot 2019-05-06 at 10.00.57.png, Screen Shot > 2019-05-07 at 1.40.47.png, Screenshot from 2019-05-06 13-45-40.png, > Screenshot from 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838825#comment-16838825 ] Jun Ohtani commented on LUCENE-8793: Hi [~Tomoko Uchida], Thanks for finding a bug. I fixed the bug you mentioned. And also changed the layout "Step by Step" checkbox. I attached [^LUCENE-8793.patch] that has the original name. Please review new patch file? Thanks > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Priority: Minor > Attachments: LUCENE-8793-2.patch, LUCENE-8793.patch, > LUCENE-8793.patch, Screen Shot 2019-05-06 at 10.00.57.png, Screen Shot > 2019-05-07 at 1.40.47.png, Screenshot from 2019-05-06 13-45-40.png, > Screenshot from 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16837742#comment-16837742 ] Tomoko Uchida commented on LUCENE-8793: --- Hi [~jun_o], thanks for updating the patch. By the way I think you should update your old patch (by uploading the patch file named "LUCENE-8793.patch"), not add a new one. See: [https://wiki.apache.org/solr/HowToContribute#JIRA_tips_.28our_issue.2Fbug_tracker.29] The layout looks fine to me. I may adjust this a little before merging it to Lucene's master branch. I saw this error when double-clicking a term (cell) in the tokenizer/token filter table. {code} [2019-05-11T11:52:10,640] ERROR (ExceptionHandler.java:39) - Index -1 out of bounds for length 5 java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 5 at jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) ~[?:?] at jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[?:?] at jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248) ~[?:?] at java.util.Objects.checkIndex(Objects.java:372) ~[?:?] at java.util.ArrayList.get(ArrayList.java:458) ~[?:?] at org.apache.lucene.luke.app.desktop.components.fragments.analysis.StepByStepAnalyzeResultPanelProvider.showAttributeValues(StepByStepAnalyzeResultPanelProvider.java:154) ~[java/:?] at org.apache.lucene.luke.app.desktop.components.fragments.analysis.StepByStepAnalyzeResultPanelProvider$ListenerFunctions.showAttributeValues(StepByStepAnalyzeResultPanelProvider.java:180) ~[java/:?] at org.apache.lucene.luke.app.desktop.components.fragments.analysis.StepByStepAnalyzeResultPanelProvider$1.mouseClicked(StepByStepAnalyzeResultPanelProvider.java:92) ~[java/:?] at java.awt.AWTEventMulticaster.mouseClicked(AWTEventMulticaster.java:278) ~[?:?] {code} When you double-click the cell, a dialog should pop up to show all token attributes. And one more small request: I'd like to locate "Step by Step" button just next to (or under) the "Test Analyzer" button. Can you please change this? > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Priority: Minor > Attachments: LUCENE-8793-2.patch, LUCENE-8793.patch, Screen Shot > 2019-05-06 at 10.00.57.png, Screen Shot 2019-05-07 at 1.40.47.png, Screenshot > from 2019-05-06 13-45-40.png, Screenshot from 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833986#comment-16833986 ] Jun Ohtani commented on LUCENE-8793: I've changed the patch for fixing 1, 3, and 4. And added charfilter's result. Also changed the layout of results. See detail in LUCENE-8793-2.patch file. !Screen Shot 2019-05-07 at 1.40.47.png|width=768,height=544! > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Priority: Minor > Attachments: LUCENE-8793-2.patch, LUCENE-8793.patch, Screen Shot > 2019-05-06 at 10.00.57.png, Screen Shot 2019-05-07 at 1.40.47.png, Screenshot > from 2019-05-06 13-45-40.png, Screenshot from 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833762#comment-16833762 ] Jun Ohtani commented on LUCENE-8793: Thanks for your comment. I will fix 1, 3, and 4. About 2, I was worry about the difference between current table and step by step table if we use vertical layout. There is a confusion. If we use vertical layout, we should change the layout current table too. I agree with you about showing charfilters result. I will try to consider layouts. > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Priority: Minor > Attachments: LUCENE-8793.patch, Screen Shot 2019-05-06 at > 10.00.57.png, Screenshot from 2019-05-06 13-45-40.png, Screenshot from > 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833542#comment-16833542 ] Tomoko Uchida commented on LUCENE-8793: --- [~jun_o]: thanks for the nice patch! I did some sanity checks locally. First of all, this works for me by the ant target: {{ant -f lucene/luke/build.xml run}} :) I noticed a few things: 1. Javadocs linter failed, so {{ant precommit}} also failed. {code:java} -documentation-lint: [echo] checking for broken html... [jtidy] FIXME: Broken HTML checks were disabled, as jtidy can't handle HTML5. [echo] Checking for broken links... [exec] [exec] Crawl/parse... [exec] [exec] Verify... [echo] Checking for missing docs... [exec] [exec] build/docs/luke/org/apache/lucene/luke/app/desktop/components/fragments/analysis/package-summary.html [exec] missing description: SimpleAnalyzeResultPanelOperator.TokensTableModel [exec] missing description: SimpleAnalyzeResultPanelProvider [exec] missing description: StepByStepAnalyzeResultPanelOperator.NamedTokensTableModel [exec] missing description: StepByStepAnalyzeResultPanelProvider [exec] [exec] Missing javadocs were found! BUILD FAILED {code} But we can delay this until all other things are done. 2. Omitting char filters is not intuitive. I think this patch is not complete without support for the charfilters, because users would expect whole analysis steps are shown in this panel. Maybe we can show the results for char filters along with those for tokenizer/tokenfilters by laying out the sub-panels vertically, not horizontally? (I have not yet checked the details. Just an idea.) Vertical layout may be also good when you add many filters like this: !Screenshot from 2019-05-06 13-45-40.png|width=724,height=212! In this case, I added a char filter and four token filters. !Screenshot from 2019-05-06 13-46-16.png|width=362,height=216! 3. "Test Analyzer Step by Step" button is visible when a "Preset" analyzer is selected. The button does not make sense with preset analyzers, because there is always only one step and the result is identical with "Test Analyzer". I think this button should not be visible (or at least disabled) when "Preset" is selected. 4. "Test Analyzer Step by Step" could be a checkbox, not a button. To me, this functionality looks like an option to change "Test Analyzer" behaviour. So I'd use a checkbox (labeled "step-by-step", for example) to switch the result panel. Do you have thoughts about this? And, I just wanted to add comments about the restriction: {quote}Current patch is only show each tokenizer/filters result independently, so the result doesn't show deletion/separation by next filter, e.g. "library" by WordDelimiterFilter is different position between other output. {quote} Except the support for charfilters as I noted above, it seems to be fine to me. This can be done by another issue (in the future.) I have not yet checked the code... will do soon. > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Priority: Minor > Attachments: LUCENE-8793.patch, Screen Shot 2019-05-06 at > 10.00.57.png, Screenshot from 2019-05-06 13-45-40.png, Screenshot from > 2019-05-06 13-46-16.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8793) Enhanced UI for CustomAnalyzer : show analysis steps
[ https://issues.apache.org/jira/browse/LUCENE-8793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833481#comment-16833481 ] Jun Ohtani commented on LUCENE-8793: !Screen Shot 2019-05-06 at 10.00.57.png|width=672,height=365! I've added new button "Test Analyzer Step By Step" for analyzing text by tokenizer & filters step by step. Also I've added new table in the lower panel for visualizing each tokenizer/filters results. If click the cell, pop up the dialog that you can see every attributes. I attached the patch file for adding this feature. Restriction: Current patch is only show each tokenizer/filters result independently, so the result doesn't show deletion/separation by next filter, e.g. "library" by WordDelimiterFilter is different position between other output. > Enhanced UI for CustomAnalyzer : show analysis steps > > > Key: LUCENE-8793 > URL: https://issues.apache.org/jira/browse/LUCENE-8793 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/luke >Reporter: Jun Ohtani >Priority: Minor > Attachments: LUCENE-8793.patch, Screen Shot 2019-05-06 at 10.00.57.png > > > This is a migrated issue from previous Luke project in GitHub: > [https://github.com/DmitryKey/luke/issues/134] > > For on-the-fly inspection / debugging, it is desirable to show the more > detailed step by step information in the Custom Analyzer UI. > This will be just like Solr's Analysis screen, > [https://lucene.apache.org/solr/guide/7_5/analysis-screen.html] > or Elasticsearch's {{_analyze}} API and Kibana's Analyzer UI. > [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html] > [https://github.com/johtani/analyze-api-ui-plugin] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org