Hi All,
Great discussion. Charles, thanks for starting it. Thanks Isabel for your
insights.
In my mind, question 4 (mission) is the most critical: what users do we want to
serve? What problems should Drill solve for these users? We've touched on this
topic in previous threads, perhaps we need to continue that discussion.
IMHO* Drill, unfortunately, did not win the "data lake" wars. Cloud vendors won
for data lakes at scale. Snowflake won for corporate data analytics as a
service. Presto won for "non-aligned" data lakes. So, where can Drill add
value? This is what folks in Silicon Valley call a "pivot."
Again, IMHO, perhaps there is a future in app data integration: the ability to
pull, and combine, data from the myriad S3 file formats and specialized DBs
that make up modern apps. Presto must see this opportunity also: they aregoing
all out to dominate this area; having added many new connectors in the last
year.
Perhaps Ted, with his wide experience talking to users and the open source
community, can offer us suggestions.
Realistically, defining a mission must be done in conjunction with point 2:
community building. If we reach out the community and pay close attention to
where we find interest, that will help identify an un- (or under-) served niche
where we add unique value. (This is, in fact, basic marketing.)
Once we determine our target "market", we'll have a better sense where to drive
the project, documentation, user experience and developer experience to satisfy
that market. Thanks to the many outstanding people who contributed to the
project over the years, Drill is actually pretty good compared to many
projects, Having a mission will tell us where we need to improve further.
So, how do we proceed?
Thanks,
- Paul
* In my *humble* opinion. I've read that many folks now read the "h" as
"honest."
On Thursday, January 30, 2020, 8:46:01 AM PST, Charles Givre
<cgi...@gmail.com> wrote:
I'd agree with Ted's thoughts. In general, my thought was that we need to
work on getting more users. More users likely will lead to more contributors,
so the goal being to make their experience as good as possible such that more
and more people will want to use Drill and tell their friends.
Also, we do need to work on the developer documentation and in general making
it easier to develop for Drill. Paul and Volodmyr have been doing great work in
cleaning up code and providing good docs for it. Again, if someone wants want
to develop for Drill and finds the code to be extremely difficult to
understand, they'll give up and do something else.
The project that Isabel mentioned as an example was the original HTTPD project,
which lacks corporate backing and yet is thriving. Anyway, I do think we
should have a continued discussion about the vision and/or mission of Drill.
-- C
> On Jan 30, 2020, at 11:18 AM, Ted Dunning <ted.dunn...@gmail.com> wrote:
>
> Igor,
>
> Good documentation and first 5-minute experience are very important, but
> not because a long-term contributor will see it and commit their spare time
> for the next five years on that basis. It is more about preventing early
> attrition of contributors who might find the project very exciting due to
> silly factors. That can easily happen if the documentation is bad because
> it increases the frustration a potential contributor feels early on. If
> they can't try the software and get something interesting, then we are
> likely to lose the battle for attention span.
>
> And frankly, it isn't just the developer that we need to attract and
> retain. A user who never contributes a line of code is part of the
> community and can easily be a net positive if they only report problems and
> occasionally tell people what they are doing.
>
>
>
> On Thu, Jan 30, 2020 at 7:00 AM Igor Guzenko <ihor.huzenko....@gmail.com>
> wrote:
>
>> Hello Charles,
>>
>> Thank you very much for starting this important discussion. These are all
>> important things but at the current moment, I don't have a clear vision
>> where we could start from. The item which is most interesting to me is the
>> second one, but I've never been involved in building an open-source
>> community, don't even know where to start. I'm not sure that just making
>> good documentation and the first impression will attract developers with
>> strong motivation to contribute. So I'm very excited to learn about
>> projects which managed to build such a community, maybe we really could
>> find some new fresh ideas about how to attract new community members.
>>
>> Thanks,
>> Igor
>>
>> On Thu, Jan 30, 2020 at 4:18 PM Charles Givre <cgi...@gmail.com> wrote:
>>
>>> Hello all,
>>> I mentioned in the Drill hangout last week that I had spoken with one of
>>> the original mentors for the Drill project (Isabel Drost-Fromm) and asked
>>> her advice about the future of Drill. To paraphrase what she told me:
>>>
>>> 1. There are two ways for open source projects to succeed. The first
>>> and riskier approach is with a single corporate sponsor. The obvious
>> risks
>>> are that since the corporate sponsor is footing the bill, they will
>>> prioritize their own needs over and sometimes against community needs.
>>> (This is not unique to Drill). The slower but less risky approach is to
>>> build a community around a project, join forces and slowly drive it
>>> forward. She pointed out that some of the Apache foundation's longest
>>> running projects were run in this way.
>>>
>>> 2. We should focus our efforts on community building: She suggested a
>>> lot of what she described as "would be obvious in retrospect" such as
>>> making sure the documentation is really solid, and having a user
>> experience
>>> in the beginning. She said we should use the resources of the Apache
>>> foundation to help publicize new releases etc. Also we should make it
>> easy
>>> to become a committer. IMHO, I would add that we really should seek out
>>> additional code reviewers as we don't have enough and PRs take a long
>> time
>>> to get approved.
>>>
>>> 3. Do a lot of what a vendor would do: Update the website and
>>> documentation to reflect things like: who is using Drill, who is offering
>>> professional support for Drill etc.
>>>
>>> 4. Define a mission: We should work to define a mission for Drill? IE
>>> Why does/should it exist and what business problem does it solve? IMHO
>> it
>>> solves a very large one, but more people need to know about it. That's
>> why
>>> I'm not giving up on it yet.
>>>
>>>
>>> @Isabel, I hope I captured the essence of what you were telling me here.
>>>
>>> Thanks everyone,
>>> --C
>>>
>>>
>>>
>>>
>>
