[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948169#comment-16948169 ] Tim Allison commented on TIKA-1334: --- At ApacheCon and Activate, some attendees at the tika-eval talks recommended Grafana or maybe Superset. > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison >Priority: Major > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898061#comment-16898061 ] Tim Allison commented on TIKA-1334: --- Apache Zeppelin might be a flexible way to visualize tika-eval's stats. We could offer example notebooks for jdbc and Solr. We might want to add d3 modules or similar for some more fun visualizations than are currently possible. > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison >Priority: Major > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022700#comment-16022700 ] Tim Allison commented on TIKA-1334: --- [~kalaspuffar], thank you for chipping in! While [~gizmo693] takes the lead on the initial contribution, one area that we could all participate in now is in the types of reports/visualizations/user stories (gah!) that would be useful. You can tell from my comparison reports which types of reports I initially thought would be useful, but I haven't included everything I've thought of, and other eyes on this would be much appreciated. If there are visualizations/info you want that the db doesn't currently support, we can open separate tickets to support those. I've opened a temporary wiki page to catalog a wish list: https://wiki.apache.org/tika/TikaEvalGUIWishlist Ping [~chrismattmann] for permissions if you'd like to participate on the wiki. Thank you, all! > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022568#comment-16022568 ] Tom Barber commented on TIKA-1334: -- Hi [~kalaspuffar] I had volunteered [~gizmo693] to do the initial framework as a way to get into web development in general and also get committing to some ASF projects, so unless there is a huge rush it would be cool if you can hang fire and let us get some code in and then we can all iterate on that. Of course this is open source and if there is a huge rush for the feature or something, then feel free to go ahead, I think it would be a good way to get a new person into committing code to Apache (plus he sits next to me so I can check the code is sane! ;) ) > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16022342#comment-16022342 ] Daniel Persson commented on TIKA-1334: -- Hi everyone. Great job on the markups. Seems great. My question is there an implementation going? Didn't see any activity in the branch or pull requests. I could contribute but it would be contra productive to work on different sides and require a merge of work later. Another tidbit of information I could share when it comes to working with JSON or a JSON server you might want to look in to testing https://github.com/typicode/json-server I usually use this server when I prototype things. You could easily generate some fake JSON and vola you have a rest API that answers queries so you could try out your frontend code before you actually implement the backend code. Seems like we are on a great start and please tell me if I can help in anyway. My first thought was to start hacking but as your already on the way there I don't want to impede your progress. Best regards Daniel > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16020071#comment-16020071 ] Tim Allison commented on TIKA-1334: --- Y. Very cool! > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16019785#comment-16019785 ] Chris A. Mattmann commented on TIKA-1334: - awesome! they look great! > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16019676#comment-16019676 ] Stephen Downie commented on TIKA-1334: -- Hi, I'm StephenTom's new guy. I met Tim at ApacheCon and got volunteered to write the UI for Tika Explore. See file attached for some mockup examples. !1-Landing page.png! Above is the splash screen/landing page. !3-Mime Types.png! This is the mime times top level page as an example. It shows the high level detail, users would be able to drill down into the specific mime type content disposition to see the most frequent encodings etc. The other attached screenshots are pretty self explanatory. Let me know what you all think & if the work is still required. Thanks > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: 1-Landing page.png, 2-File Types.png, 3-Mime Types.png, > 4-Detected Extensions.png, 5-Conflicts between actual and detected > extension.png, static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16016303#comment-16016303 ] Tom Barber commented on TIKA-1334: -- my new guy is looking for an excuse to get started in programming I'll point him in this direction if you don't mind it starting basic and iterating > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998940#comment-15998940 ] Chris A. Mattmann commented on TIKA-1334: - thanks! > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998387#comment-15998387 ] Tim Allison commented on TIKA-1334: --- Added new TIKA-1334 branch for work on this. > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996690#comment-15996690 ] Tim Allison commented on TIKA-1334: --- New dev branch in ASF's repo for work on this? > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[
https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994941#comment-15994941
]
Tyler Palsulich commented on TIKA-1334:
---
The format should probably be in the form:
{noformat}
[
{
"mime-type": "something",
"count": 1234,
"version": "a"
},
{
"mime-type": "something",
"count": 4321,
"version": "b"
},
...
]
{noformat}
> Add presentation layer for results of each run
> --
>
> Key: TIKA-1334
> URL: https://issues.apache.org/jira/browse/TIKA-1334
> Project: Tika
> Issue Type: Sub-task
> Components: cli, general, server
>Reporter: Tim Allison
> Attachments: static_stats.zip
>
>
> If I'm doing this, it'll probably be vintage mid-90s html. If someone with
> some .js kung-fu wants to take this, please do.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994920#comment-15994920 ] Chris A. Mattmann commented on TIKA-1334: - for the first proposal, see: http://drat.dyndns.org:8080/dratviz/ [~karanjeets] from USC IRDS made it - he would be happy to help. > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[
https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994691#comment-15994691
]
Tim Allison commented on TIKA-1334:
---
2 is the eventual goal with the "out-of-the-box" configuration running a local
server on localhost, all locked down to local host.
We can then add configurability to allow people to run a true server.
We can call the server "Explore", so tika-eval will have {{Profile}},
{{Compare}}, {{Report}}, {{StartDB}} and {{Explore}}.
But first, static json. [~tpalsulich], if you give me an example of your
desired json, I'll add that to "Report".
> Add presentation layer for results of each run
> --
>
> Key: TIKA-1334
> URL: https://issues.apache.org/jira/browse/TIKA-1334
> Project: Tika
> Issue Type: Sub-task
> Components: cli, general, server
>Reporter: Tim Allison
> Attachments: static_stats.zip
>
>
> If I'm doing this, it'll probably be vintage mid-90s html. If someone with
> some .js kung-fu wants to take this, please do.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994689#comment-15994689 ] Tim Allison commented on TIKA-1334: --- Yes, yes and yes. 3 is a non-starter from my perspective. I'm happy to start with static json, and I can create a json writer for the reports. Proposal for first chart: pie chart of mime-types? > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994688#comment-15994688 ] Tim Allison commented on TIKA-1334: --- >From [~tpalsulich] on twitter -- let's start with one report at a time. > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[
https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994687#comment-15994687
]
Tim Allison commented on TIKA-1334:
---
>From [~tpalsulich] on the mailing list
{noformat}
I also like d3. In general, I think we are on the same page the best option is
a web based UI.
I see a few options to get data into the frontend:
1. Static JSON
2. JSON from a server (meaning the server runs queries (either built by the
client or the server)) 3. Load a local DB (meaning the client runs queries)
>From some quick searching, 3 seems like it has poor support. I could be wrong.
1 and 2 are clearly related. If we have a working application with static JSON,
changing it to use served JSON should be straightforward (from a Java server,
probably). Static JSON will be faster than live queries, but I don't know how
long the queries take. The polar project seems to hard code queries and provide
an interface to manually enter more.
Static JSON seems easiest to get started. What do you think?
{noformat}
> Add presentation layer for results of each run
> --
>
> Key: TIKA-1334
> URL: https://issues.apache.org/jira/browse/TIKA-1334
> Project: Tika
> Issue Type: Sub-task
> Components: cli, general, server
>Reporter: Tim Allison
> Attachments: static_stats.zip
>
>
> If I'm doing this, it'll probably be vintage mid-90s html. If someone with
> some .js kung-fu wants to take this, please do.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312155#comment-14312155 ] Tim Allison commented on TIKA-1334: --- No, not pretty. This is a first step. The code for tika-eval is still very rough around the edges, but it is on my github site under branch TIKA-1302. In my current design, tika-eval will have three primary chunks of code: * A profiler which will run through a directory of output and populate a database and generate reports similar to the attached but for a single run. * A directory comparison tool that will run through a pair of directories, and run comparisons on a file pair-wise level. This will generate static reports similar to the attached, but this will also populate a database that we can use in an interactive ui. * Some kind of interactive ui that will allow users to drill down and view reports, summary statistics, output diffs and source files. I just transitioned to h2 for the db, and I was quite impressed with the fairly flat memory consumption even at 1M files. > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TIKA-1334) Add presentation layer for results of each run
[ https://issues.apache.org/jira/browse/TIKA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310884#comment-14310884 ] Tyler Palsulich commented on TIKA-1334: --- This dump actually seems pretty good... Not very pretty, and there's a lot of info to digest, but how is it generated? > Add presentation layer for results of each run > -- > > Key: TIKA-1334 > URL: https://issues.apache.org/jira/browse/TIKA-1334 > Project: Tika > Issue Type: Sub-task > Components: cli, general, server >Reporter: Tim Allison > Attachments: static_stats.zip > > > If I'm doing this, it'll probably be vintage mid-90s html. If someone with > some .js kung-fu wants to take this, please do. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
