Re: Where are the Linked Data Driven Smart Agents (Bots) ?
HI Krzysztof, > this is all about finding the right balance Definitely—but I have the feeling the balance is currently tipped very much to one side (and perhaps not the side that delivers the most urgent components for the SemWeb). > as we also do not want to have tons of 'ideas' > papers without any substantial content or proof of concept Mere ideas would indeed not be sufficient; but even papers with substantial content and/or a proof of concept will have a difficult time getting accepted if there is no evaluation that satisfies the reviewers. (And, lacking a framework to evaluate evaluations, I see people typically choosing for things they know, hence why incremental research gets accepted easily.) Best, Ruben
Re: Where are the Linked Data Driven Smart Agents (Bots) ?
As such, it is hard to publish a paper on this at any of the main venues (ISWC / ESWC / …). This discourages working on such themes. Hence, I see much talent and time going to incremental research, which is easy to evaluate well, but not necessarily as ground-breaking. Yes! I could not agree more. On the other hand, this is all about finding the right balance as we also do not want to have tons of 'ideas' papers without any substantial content or proof of concept. I remember that there was an ISWC session some years ago that tried to introduce such a 'bold ideas' track. Krzysztof On 07/06/2016 09:38 AM, Ruben Verborgh wrote: Hi, This is a very important question for our community, given that smart agents once were an important theme. Actually, the main difference we could bring with the SemWeb is that our clients could be decentralized and actually run on the client side, in contrast to others. One of the main problems I see is how our community (now particularly thinking about the scientific subgroup) receives submissions of novel work. We have evolved into an extremely quantitative-oriented view, where anything that can be measured with numbers is largely favored over anything that cannot. Given that the smart agents / bots field is quite new, we don't know the right evaluation metrics yet. As such, it is hard to publish a paper on this at any of the main venues (ISWC / ESWC / …). This discourages working on such themes. Hence, I see much talent and time going to incremental research, which is easy to evaluate well, but not necessarily as ground-breaking. More than a decade of SemWeb research has mostly brought us intelligent servers, but not yet the intelligent clients we wanted. So perhaps we should phrase the question more broadly: how can we as a community be more open to novel and disruptive technologies? Best, Ruben -- Krzysztof Janowicz Geography Department, University of California, Santa Barbara 4830 Ellison Hall, Santa Barbara, CA 93106-4060 Email: j...@geog.ucsb.edu Webpage: http://geog.ucsb.edu/~jano/ Semantic Web Journal: http://www.semantic-web-journal.net
Re: Where are the Linked Data Driven Smart Agents (Bots) ?
On 7 July 2016 at 20:49, Juan Sequeda wrote: > > > On Thu, Jul 7, 2016 at 12:51 PM, Ruben Verborgh > wrote: > >> Hi Juan, >> >> Seems like we mostly agree—short remarks below. >> >> > One thing is science. Another is engineering. >> >> Perhaps we need Semantic Web Engineering conferences then as well! >> > > That's why you have developer conferences, etc. ISWC this year had a > Applications track. And it doesn't have to be semantic web specific. > > >> >> > If we don't know the right evaluation metrics (I agree with you that we >> don't), then that is the current challenge we, as a semantic web scientific >> community, have to tackle. >> >> Indeed, but I've found the scientific community to be not so open to new >> evaluation metrics either. There is insufficient agreement on (and too >> limited knowledge of) the right scientific methodology to tackle such novel >> problems. >> > > Well, I don't know what you have proposed. But it seems that you haven't > made a convincing argument :P (if you want to discuss offline what you are > doing, send me an email) > > >> >> > It shouldn't discourage you... on the contrary, it should encourage you >> to identify novel ways to evaluate what you are doing and convince the >> community why it is important. >> >> The trouble is you don't have to convince the entire community (with whom >> you can have an open dialog), but a tiny set of anonymous reviewers (for >> whom the known paths are often easier to judge). >> > > Paper reviewing... that is different topic I don't want to get into here. > > >> My remark was precisely that convincing is hard once you move away from >> the known paths. >> >> So the scientific community, which is a large part of the total Semantic >> Web community, might in that sense be hampering real novelty—from science >> and engineering alike, whichever might be the difference. >> > > There is a clear difference between Science and Engineering. That is my > point. Science is about understanding what is unknown. In this case, it is > not well known how to evaluate new types of systems. That is what needs to > be studied. We need to figure out how to evaluate and evaluation. It is a > bit meta. Jim Hendler stated this 7 years ago and to the best of my > knowledge, this is an area that hasn't been tackled (PhD thesis anybody?) > > If your goal is to get something out there and for it to be used, then why > do you bother spending time publishing papers. Look at all the open source > projects changing the world, with very little to no scientific > publications. > > Just do what makes you happy and be the best at it. Strive for excellence! > I know you are :) > I think it would be nice to have a common ground across the field. e.g. - Everyone has a profile page that is machine readable (so many people still dont!) - Beginning with that profile page, cross origin links are made, to create new connections, with friends, colleagues, people, places, things. - The social graph is both browsable and writable to, so that changes can be made, propagated and viewed by others. - Each individual research project, feeds into this graph, augments it, improves it, creates new possibilities. In short, why not spend 20+% of our time using the semantic web in collaboration with others to make it come alive? > > >> Best, >> >> Ruben > > >
Re: Where are the Linked Data Driven Smart Agents (Bots) ?
On Thu, Jul 7, 2016 at 12:51 PM, Ruben Verborgh wrote: > Hi Juan, > > Seems like we mostly agree—short remarks below. > > > One thing is science. Another is engineering. > > Perhaps we need Semantic Web Engineering conferences then as well! > That's why you have developer conferences, etc. ISWC this year had a Applications track. And it doesn't have to be semantic web specific. > > > If we don't know the right evaluation metrics (I agree with you that we > don't), then that is the current challenge we, as a semantic web scientific > community, have to tackle. > > Indeed, but I've found the scientific community to be not so open to new > evaluation metrics either. There is insufficient agreement on (and too > limited knowledge of) the right scientific methodology to tackle such novel > problems. > Well, I don't know what you have proposed. But it seems that you haven't made a convincing argument :P (if you want to discuss offline what you are doing, send me an email) > > > It shouldn't discourage you... on the contrary, it should encourage you > to identify novel ways to evaluate what you are doing and convince the > community why it is important. > > The trouble is you don't have to convince the entire community (with whom > you can have an open dialog), but a tiny set of anonymous reviewers (for > whom the known paths are often easier to judge). > Paper reviewing... that is different topic I don't want to get into here. > My remark was precisely that convincing is hard once you move away from > the known paths. > > So the scientific community, which is a large part of the total Semantic > Web community, might in that sense be hampering real novelty—from science > and engineering alike, whichever might be the difference. > There is a clear difference between Science and Engineering. That is my point. Science is about understanding what is unknown. In this case, it is not well known how to evaluate new types of systems. That is what needs to be studied. We need to figure out how to evaluate and evaluation. It is a bit meta. Jim Hendler stated this 7 years ago and to the best of my knowledge, this is an area that hasn't been tackled (PhD thesis anybody?) If your goal is to get something out there and for it to be used, then why do you bother spending time publishing papers. Look at all the open source projects changing the world, with very little to no scientific publications. Just do what makes you happy and be the best at it. Strive for excellence! I know you are :) > Best, > > Ruben
Re: Where are the Linked Data Driven Smart Agents (Bots) ?
Hi Juan, Seems like we mostly agree—short remarks below. > One thing is science. Another is engineering. Perhaps we need Semantic Web Engineering conferences then as well! > If we don't know the right evaluation metrics (I agree with you that we > don't), then that is the current challenge we, as a semantic web scientific > community, have to tackle. Indeed, but I've found the scientific community to be not so open to new evaluation metrics either. There is insufficient agreement on (and too limited knowledge of) the right scientific methodology to tackle such novel problems. > It shouldn't discourage you... on the contrary, it should encourage you to > identify novel ways to evaluate what you are doing and convince the community > why it is important. The trouble is you don't have to convince the entire community (with whom you can have an open dialog), but a tiny set of anonymous reviewers (for whom the known paths are often easier to judge). My remark was precisely that convincing is hard once you move away from the known paths. So the scientific community, which is a large part of the total Semantic Web community, might in that sense be hampering real novelty—from science and engineering alike, whichever might be the difference. Best, Ruben
CFPs and the lists
Hmmm. So I am enjoying the new regime without CfPs on the LOD list (many thanks, Phil!). However, I now find myself thinking I will unsubscribe from the SemWeb list, since it is almost all CfPs, few, if any, of which I want. I think this may be an unintended consequence (although probably predictable) - losing people from SemWeb. There isn’t really a question here - I just thought I would report it. If there is a question: dare I suggest that now things have settled down, and we can see how things are working, that we might want to revisit the idea of having a separate list for CfPs, and reclaim the SemWeb list for discussion? (Sorry Phil?) Or is the answer that I simply set mail filters and carry on? Best Hugh
Re: Where are the Linked Data Driven Smart Agents (Bots) ?
Ruben, One thing is science. Another is engineering. Part of the scientific process is defining an experiment and doing the evaluation. If we don't know the right evaluation metrics (I agree with you that we don't), then that is the current challenge we, as a semantic web scientific community, have to tackle. It shouldn't discourage you... on the contrary, it should encourage you to identify novel ways to evaluate what you are doing and convince the community why it is important. In my opinion, evaluation of systems on the web is different that what the CS community has been used to (CS is a young science). Evaluation has been on run time, space consumed, precision, recall, sound, completeness. On the web, those may not be the aspects we want to measures. In the words of Jim Hendler: “You want a good thesis? IR is based on precision and recall and the minute you add semantics, it is a meaningless feature. Logic is based on soundness and completeness. We don’t want soundness and completeness. We want a few good answers quickly.” – Prof. James A. Hendler, 2009, on the topic of answering queries over the Semantic Web. https://www.youtube.com/watch?v=sbmMxzOeZ-4 Here is a short position paper that Olaf Hartig and I wrote 5 years ago (time flies when you are having fun): Towards a Query Language for the Web of Data (A Vision Paper). http://arxiv.org/abs/1110.3017 My 2cts -- Juan Sequeda, Ph.D +1-575-SEQ-UEDA www.juansequeda.com On Wed, Jul 6, 2016 at 11:38 AM, Ruben Verborgh wrote: > Hi, > > This is a very important question for our community, > given that smart agents once were an important theme. > Actually, the main difference we could bring with the SemWeb > is that our clients could be decentralized > and actually run on the client side, in contrast to others. > > One of the main problems I see is how our community > (now particularly thinking about the scientific subgroup) > receives submissions of novel work. > We have evolved into an extremely quantitative-oriented view, > where anything that can be measured with numbers > is largely favored over anything that cannot. > > Given that the smart agents / bots field is quite new, > we don't know the right evaluation metrics yet. > As such, it is hard to publish a paper on this > at any of the main venues (ISWC / ESWC / …). > This discourages working on such themes. > > Hence, I see much talent and time going to > incremental research, which is easy to evaluate well, > but not necessarily as ground-breaking. > More than a decade of SemWeb research > has mostly brought us intelligent servers, > but not yet the intelligent clients we wanted. > > So perhaps we should phrase the question more broadly: > how can we as a community be more open > to novel and disruptive technologies? > > Best, > > Ruben >