Re: TODO list (was Re: Contributing with code)
On 01/03/2018 09:00 AM, Alvaro Herrera wrote: Joshua D. Drake wrote: There's already https://postgresql.uservoice.com/forums/21853-general which seems to work pretty well. Here's the list of completed items: https://postgresql.uservoice.com/forums/21853-general?status_id=124172 Well that's interesting, I have never heard of that. Is that something we want to promote more? JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stated, opinions are my own. *
Re: TODO list (was Re: Contributing with code)
Joshua D. Drake wrote: > Heck we could go a step further and actually allow (authenticated) voting on > various features. This would provide the community the ability to more > easily interact with -hackers on various features that would be desirable. There's already https://postgresql.uservoice.com/forums/21853-general which seems to work pretty well. Here's the list of completed items: https://postgresql.uservoice.com/forums/21853-general?status_id=124172 -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: TODO list (was Re: Contributing with code)
On 1/3/18 11:10, Alvaro Herrera wrote:
> This text was added by [1] as saying:
> > This list contains '''some known PostgreSQL bugs and feature
> > requests''' and we hope it contains all such
> (before this, it said "all known Pg bugs" which seemed too optimistic,
> so the correction was in a good direction) and later amended by [2] to
> the current wording ("This list contains '''known PostgreSQL bugs and
> feature requests''' and we hope it is complete"). I think [1]'s wording
> was okay, but the other one not so much.
>
> I rewrote the text. I thought about presenting my draft here first, but
> hey, it's a wiki.
I went through it just now as well. I have cleaned up the formatting a
bit and removed some entries that were done.
It's not bad really, as long as the intro text is more appropriate.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: TODO list (was Re: Contributing with code)
On 01/03/2018 07:49 AM, Jeff Janes wrote: O.k. what does it tell us though? Is it a resource issue? Is it a barrier of entry issue? Lack of ownership/ruthlessness. While I can edit it to remove items that don't seem desirable (or comprehensible, or whatever) I'm not likely to do so, unless I'm the one who added it in the first place. Maybe it made more sense or was more important to someone else, like the person who added it. At one time many of the items didn't have links to the relevant email discussions (or more detailed wiki pages of their own), so those would have been good targets for purging but I think Bruce hunted down and added links for most of them. Another problem is that wikimedia doesn't have a "git blame" like feature. I've been frustrated before trying to figure out who added an item and when, so I could research it a bit more. It seems to me that this is rather easily solved with an issue tracker or enhancement to the commitfest app. With an issue tracker we just set a status of "wishlist" and then set up a simple public report that allows people to see wishlist items. Plus we get the benefit of an issue tracker. The commitfest app could work in a similar fashion. The commitfest app could be replaced by an issue tracker to, but we as a community tend to suffer from NIH, so it might be less of a battle to just adjust the commitfest app. Either one of these options allows to consolidation of tools as well as information portals both of which would help the community. Heck we could go a step further and actually allow (authenticated) voting on various features. This would provide the community the ability to more easily interact with -hackers on various features that would be desirable. (I am not suggesting that the voting dictate our direction but to allow feedback on new and interesting ideas that show community support) Of course that takes resources. There have been discussions of getting an issue tracker in the past but the priority has appeared to drop to the wayside. Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stated, opinions are my own. *
Re: TODO list (was Re: Contributing with code)
I think deleting the TODO list is a bad idea -- it contains very useful
pointers to previous discussion on hard topics.
Jeff Janes wrote:
> On Tue, Jan 2, 2018 at 2:48 PM, Robert Haas wrote:
> > It also has a note at the top saying we think it's complete, but we
> > don't think that, or I don't think it anyway.
>
> Yeah, I don't care for that, either.
This text was added by [1] as saying:
> This list contains '''some known PostgreSQL bugs and feature
> requests''' and we hope it contains all such
(before this, it said "all known Pg bugs" which seemed too optimistic,
so the correction was in a good direction) and later amended by [2] to
the current wording ("This list contains '''known PostgreSQL bugs and
feature requests''' and we hope it is complete"). I think [1]'s wording
was okay, but the other one not so much.
I rewrote the text. I thought about presenting my draft here first, but
hey, it's a wiki.
[1] https://wiki.postgresql.org/index.php?title=Todo&diff=17840&oldid=17669
[2] https://wiki.postgresql.org/index.php?title=Todo&diff=17961&oldid=17877
--
Álvaro Herrerahttps://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: TODO list (was Re: Contributing with code)
On Tue, Jan 2, 2018 at 6:42 PM, Joshua D. Drake wrote: > On 01/02/2018 11:17 AM, Robert Haas wrote: > >> On Sun, Dec 31, 2017 at 2:31 PM, Peter Geoghegan wrote: >> >>> On Sun, Dec 31, 2017 at 10:42 AM, Tom Lane wrote: >>> If we're not going to maintain/curate it properly, I agree it's not worth keeping it around. But I'd rather see somebody put some effort into it ... >>> If somebody was going to resolve to put some effort into maintaining >>> it to a high standard then it probably would have happened already. >>> The fact that it hasn't happened tells us plenty. >>> >> +1, and well said. >> > > O.k. what does it tell us though? Is it a resource issue? Is it a barrier > of entry issue? Lack of ownership/ruthlessness. While I can edit it to remove items that don't seem desirable (or comprehensible, or whatever) I'm not likely to do so, unless I'm the one who added it in the first place. Maybe it made more sense or was more important to someone else, like the person who added it. At one time many of the items didn't have links to the relevant email discussions (or more detailed wiki pages of their own), so those would have been good targets for purging but I think Bruce hunted down and added links for most of them. Another problem is that wikimedia doesn't have a "git blame" like feature. I've been frustrated before trying to figure out who added an item and when, so I could research it a bit more. > What does deleting it solve? What problems (and there is a very large > obvious one) are caused by deleting it? > > Right now, the TODO list is the "only" portal to "potential" things we > "might" want. If we delete it we are just creating yet another barrier of > entry to potential contribution. I think we need to consider an alternative > solution because of that. > There are various "roadmaps" floating around (wiki and elsewhere), but they aren't very prominent or easy to find. They seem to mostly be minutes from meetings, but you wouldn't know to look for them if you weren't at the meeting. Cheers, Jeff
Re: TODO list (was Re: Contributing with code)
On Tue, Jan 2, 2018 at 2:48 PM, Robert Haas wrote: > On Sun, Dec 31, 2017 at 2:02 PM, David G. Johnston > wrote: > > It probably needs three sub-sections. Fist the raw ideas put forth by > > people not capable of implementation but needing capabilities; these get > > moved to one of two sections: ideas that have gotten some attention by > core > > that have merit but don't have development interest presently; and one > like > > this that have gotten the some attention and that core doesn't feel > would be > > worth maintaining even if someone was willing to develop it. We already > > have this in practice but maybe a bit more formality would help. > > > > I'm not seeing that having it, even if incorrect, does harm. > > It causes people to waste time developing features we don't want. > We don't want them at all, or we just don't want a naive implementation of them? If we don't want them at all, then surely we should remove those items. Or move them to the bottom of the page, where there is a section just for such things. That way people can at least see that it has been considered and rejected. And for things that we do want, it is nice to have links to the emails of where it was discussed/attempted before. This is especially useful because searching the archives is very inefficient due to nearly every word you want to search on being a stop word or a very common word with many meanings. It is much easier to find it on the todo list if it is there than to search the archives. > > It also has a note at the top saying we think it's complete, but we > don't think that, or I don't think it anyway. > Yeah, I don't care for that, either. Cheers, Jeff
Re: TODO list (was Re: Contributing with code)
On 01/03/2018 03:50 AM, David Rowley wrote: > On 3 January 2018 at 13:12, Patrick Krecker wrote: >> As a person looking to become a postgres contributor, perhaps I can >> offer some perspective on this. I think there is value in providing >> *some* starting point for new contributors in the form of concrete >> problems to solve. The value I hope to extract from the time spent on >> my first feature comes mostly from the learning experience and not >> from the acceptance of the feature itself. I would not be upset if my >> work was never accepted as long as I understand why. I expect most >> people picking features at random from a TODO list would have a >> similar outlook on their first contribution. > > I agree with this. It was about 10 years ago when I first looked at > the TODO list and thought "I could do that!", so I did. I got lucky > and it was accepted, but without that list, I'd probably never have > written the patch and probably not be here today. I, too, came in through the TODO list. -- Vik Fearing +33 6 46 75 15 36 http://2ndQuadrant.fr PostgreSQL : Expertise, Formation et Support
Re: TODO list (was Re: Contributing with code)
On 3 January 2018 at 13:12, Patrick Krecker wrote: > As a person looking to become a postgres contributor, perhaps I can > offer some perspective on this. I think there is value in providing > *some* starting point for new contributors in the form of concrete > problems to solve. The value I hope to extract from the time spent on > my first feature comes mostly from the learning experience and not > from the acceptance of the feature itself. I would not be upset if my > work was never accepted as long as I understand why. I expect most > people picking features at random from a TODO list would have a > similar outlook on their first contribution. I agree with this. It was about 10 years ago when I first looked at the TODO list and thought "I could do that!", so I did. I got lucky and it was accepted, but without that list, I'd probably never have written the patch and probably not be here today. I'd say, anyone who looks at the TODO list, picks something and implements it is doing it for a personal challenge and maybe a foot in a door. We should not make doing that any harder for people. For all the others who are implementing things because they need PostgreSQL to that, then they've not used the TODO list for that so is not a concern of this thread. I think the warning that's on the TODO list [1] is a great idea. Perhaps it should be bigger or more verbose. [1] https://wiki.postgresql.org/wiki/Todo -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Re: TODO list (was Re: Contributing with code)
Greetings, * Patrick Krecker (pkrec...@gmail.com) wrote: > As a person looking to become a postgres contributor, perhaps I can > offer some perspective on this. I think there is value in providing > *some* starting point for new contributors in the form of concrete > problems to solve. The value I hope to extract from the time spent on > my first feature comes mostly from the learning experience and not > from the acceptance of the feature itself. I would not be upset if my > work was never accepted as long as I understand why. I expect most > people picking features at random from a TODO list would have a > similar outlook on their first contribution. I really appreciate this view-point on it and think that it's definitely a good one to have, though I don't expect everyone to share it. While I tend to agree that the TODO list is valuable, what I don't think it has that is really key is any notion around 'level of difficulty' or 'who you can talk to about trying to address this item'. We had a rather successful Google Summer of Code in 2017 and I don't think that any of the projects accepted were ever on the TODO list. In running GSoC last year and reading through all of the documentation provided by Google about how to run GSoC successfully and what they're looking for in terms of a project list made me realize that the areas I mention above are really key to success. The projects proposed through GSoC are, necessairly, constrained to what can be done in a single summer (or, at least, serious progress being made), but that's actually a good thing because it means that the projects are sized well- large enough to be interesting and serious features while small enough for someone relatively new to the community. The "easy" things tend to be done by regular hackers, the really hard stuff is too big for a beginner to start with. To that end, let me just say that I strongly encourage anyone who is interested in hacking on PG to please feel free to review what's on the GSoC 2018 page and feel free to reach out to me if you're interested in working on it, even if you aren't planning to participate in GSoC 2018. We can come up with other things for GSoC students to work on, I believe, even if half of the items currently listed there get picked up on by non-GSoC individuals. For the regular hackers, when you see something that's really hard on the TODO list, please mark it as such instead of just deciding that the list is useless. If we could quantify the level of effort required, that'd be even better. Ultimately, I think it'd be fantastic to have a wiki page for each item on the TODO list that describes the project itself, what's been discussed (with links to those discussions) and what areas could use more exploration. The way forward, at least from my perspective, is to try and improve the list, not to throw it away. I'd love to, one day, be able to go through the TODO list and find the links to GSoC-sized projects to build the yearly GSoC page with instead of having to make it an independent effort. Lastly, as a reminder, we will be submitting for GSoC 2018 soon, so please put your GSoC-sized ideas on the GSoC 2018 page: https://wiki.postgresql.org/wiki/GSoC_2018 and feel free to put them on the TODO list too... Thanks! Stephen signature.asc Description: PGP signature
Re: TODO list (was Re: Contributing with code)
On Tue, Jan 2, 2018 at 3:42 PM, Joshua D. Drake wrote: > On 01/02/2018 11:17 AM, Robert Haas wrote: >> >> On Sun, Dec 31, 2017 at 2:31 PM, Peter Geoghegan wrote: >>> >>> On Sun, Dec 31, 2017 at 10:42 AM, Tom Lane wrote: If we're not going to maintain/curate it properly, I agree it's not worth keeping it around. But I'd rather see somebody put some effort into it ... >>> >>> If somebody was going to resolve to put some effort into maintaining >>> it to a high standard then it probably would have happened already. >>> The fact that it hasn't happened tells us plenty. >> >> +1, and well said. > > > O.k. what does it tell us though? Is it a resource issue? Is it a barrier of > entry issue? What does deleting it solve? What problems (and there is a very > large obvious one) are caused by deleting it? > > Right now, the TODO list is the "only" portal to "potential" things we > "might" want. If we delete it we are just creating yet another barrier of > entry to potential contribution. I think we need to consider an alternative > solution because of that. > > Thanks, > > JD > > > -- > Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc > > PostgreSQL centered full stack support, consulting and development. > Advocate: @amplifypostgres || Learn: https://postgresconf.org > * Unless otherwise stated, opinions are my own. * > > As a person looking to become a postgres contributor, perhaps I can offer some perspective on this. I think there is value in providing *some* starting point for new contributors in the form of concrete problems to solve. The value I hope to extract from the time spent on my first feature comes mostly from the learning experience and not from the acceptance of the feature itself. I would not be upset if my work was never accepted as long as I understand why. I expect most people picking features at random from a TODO list would have a similar outlook on their first contribution.
Re: TODO list (was Re: Contributing with code)
On 01/02/2018 11:17 AM, Robert Haas wrote: On Sun, Dec 31, 2017 at 2:31 PM, Peter Geoghegan wrote: On Sun, Dec 31, 2017 at 10:42 AM, Tom Lane wrote: If we're not going to maintain/curate it properly, I agree it's not worth keeping it around. But I'd rather see somebody put some effort into it ... If somebody was going to resolve to put some effort into maintaining it to a high standard then it probably would have happened already. The fact that it hasn't happened tells us plenty. +1, and well said. O.k. what does it tell us though? Is it a resource issue? Is it a barrier of entry issue? What does deleting it solve? What problems (and there is a very large obvious one) are caused by deleting it? Right now, the TODO list is the "only" portal to "potential" things we "might" want. If we delete it we are just creating yet another barrier of entry to potential contribution. I think we need to consider an alternative solution because of that. Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stated, opinions are my own. *
Re: TODO list (was Re: Contributing with code)
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Dec 31, 2017 at 1:51 PM, Stephen Frost wrote: > > The todo entry even talks about why it's difficult to do and what the > > expected way to go about doing it is (that is, connect to each database > > that has objects in the tablespace and query it to find out what's in > > the tablespace). Craig's suggestion is an interesting alternative way > > though and I'm not sure that it'd be all that bad, but it would be > > limited to catalog tables. > > I think it'd be pretty bad. There's nothing in the system that > actually guarantees that the system catalog structure matches across > every database. Of course, if you change structural properties, then > the system will probably crash, but attstorage and attacl values could > be different, as could relfilenode, relpages, reltuples, > relallvisible, relfroxenxid, relminmxid, and relacl. I don't think > it's wise to let this work on the theory that none of that stuff > matters. Even if that's true, or can be made true with a crowbar, > it's a fragile assumption that might turn false in the future. I'm surprised to hear it described as fragile when I would certainly expect a huge amount of push-back from the developer community if someone suggested making template1 have a different catalog structure from the other databases for some reason. This seems more like an unwritten rule to me than a 'it happens to work that way' kind of thing. > > If we'd extend the system to allow transparent usage of postgres_fdw to > > access other databases which are part of the same cluster, then this > > could end up being much simpler (eg: select * from > > otherdatabase.pg_catalog.pg_class ...). > > It would probably be better to use background workers for this than > postgres_fdw to avoid for example making sure you can authenticate, > but even then this is a pretty significant body of work for what I'd > consider a fairly marginal benefit. We could address the authentication issue, I believe, internally such that users wouldn't have to actually worry about it (perhaps a special entry in pg_hba.conf and something done through shared memory), which would definitely be part of the point- what I was trying to get at above is that we could possibly solve this in a much more general way by supporting, more-or-less transparently, cross-database queries, which would be a terribly useful feature, imv. I agree that it'd be a significant body of work, but we would gain a great deal more than just the ability to query the catalog of other databases and that seems quite worthwhile to me. Thanks! Stephen signature.asc Description: PGP signature
Re: TODO list (was Re: Contributing with code)
On Sun, Dec 31, 2017 at 2:02 PM, David G. Johnston wrote: > It probably needs three sub-sections. Fist the raw ideas put forth by > people not capable of implementation but needing capabilities; these get > moved to one of two sections: ideas that have gotten some attention by core > that have merit but don't have development interest presently; and one like > this that have gotten the some attention and that core doesn't feel would be > worth maintaining even if someone was willing to develop it. We already > have this in practice but maybe a bit more formality would help. > > I'm not seeing that having it, even if incorrect, does harm. It causes people to waste time developing features we don't want. It also has a note at the top saying we think it's complete, but we don't think that, or I don't think it anyway. It's basically disinformation. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Re: TODO list (was Re: Contributing with code)
On Sun, Dec 31, 2017 at 1:51 PM, Stephen Frost wrote: > The todo entry even talks about why it's difficult to do and what the > expected way to go about doing it is (that is, connect to each database > that has objects in the tablespace and query it to find out what's in > the tablespace). Craig's suggestion is an interesting alternative way > though and I'm not sure that it'd be all that bad, but it would be > limited to catalog tables. I think it'd be pretty bad. There's nothing in the system that actually guarantees that the system catalog structure matches across every database. Of course, if you change structural properties, then the system will probably crash, but attstorage and attacl values could be different, as could relfilenode, relpages, reltuples, relallvisible, relfroxenxid, relminmxid, and relacl. I don't think it's wise to let this work on the theory that none of that stuff matters. Even if that's true, or can be made true with a crowbar, it's a fragile assumption that might turn false in the future. > If we'd extend the system to allow transparent usage of postgres_fdw to > access other databases which are part of the same cluster, then this > could end up being much simpler (eg: select * from > otherdatabase.pg_catalog.pg_class ...). It would probably be better to use background workers for this than postgres_fdw to avoid for example making sure you can authenticate, but even then this is a pretty significant body of work for what I'd consider a fairly marginal benefit. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Re: TODO list (was Re: Contributing with code)
On Sun, Dec 31, 2017 at 2:31 PM, Peter Geoghegan wrote: > On Sun, Dec 31, 2017 at 10:42 AM, Tom Lane wrote: >> If we're not going to maintain/curate it properly, I agree it's not >> worth keeping it around. But I'd rather see somebody put some effort >> into it ... > > If somebody was going to resolve to put some effort into maintaining > it to a high standard then it probably would have happened already. > The fact that it hasn't happened tells us plenty. +1, and well said. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Re: TODO list (was Re: Contributing with code)
On Sun, Dec 31, 2017 at 10:42 AM, Tom Lane wrote: > If we're not going to maintain/curate it properly, I agree it's not > worth keeping it around. But I'd rather see somebody put some effort > into it ... If somebody was going to resolve to put some effort into maintaining it to a high standard then it probably would have happened already. The fact that it hasn't happened tells us plenty. -- Peter Geoghegan
Re: TODO list (was Re: Contributing with code)
On Sun, Dec 31, 2017 at 11:42 AM, Tom Lane wrote: > Robert Haas writes: > > Also, let's delete the TODO list. People keep using it as a source of > > project ideas, and that's bad. > > If we're not going to maintain/curate it properly, I agree it's not > worth keeping it around. But I'd rather see somebody put some effort > into it ... > It probably needs three sub-sections. Fist the raw ideas put forth by people not capable of implementation but needing capabilities; these get moved to one of two sections: ideas that have gotten some attention by core that have merit but don't have development interest presently; and one like this that have gotten the some attention and that core doesn't feel would be worth maintaining even if someone was willing to develop it. We already have this in practice but maybe a bit more formality would help. I'm not seeing that having it, even if incorrect, does harm. Those who would develop from it should be using it as a conversation starter and even if it is discovered that the feature is no longer desirable or applicable that conversation will have been worthwhile to that person and probably many others browsing these mailing lists. I do think that all of the commit-fest managers (and release note writers/reviewers) for the release year could be asked/reminded to at least skim over the ToDo list at the end of their session and see whether anything was addressed by one of the commits that went in during their tenure and remove (or update) them. A thread started by "hey, lets talk about this ToDo list item" is one that reinforces the value of having such a list - and at worse should result in that particular entry being updated in some fashion. David J.
Re: TODO list (was Re: Contributing with code)
Tom, Robert, all, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > Also, let's delete the TODO list. People keep using it as a source of > > project ideas, and that's bad. > > If we're not going to maintain/curate it properly, I agree it's not > worth keeping it around. But I'd rather see somebody put some effort > into it ... I don't see anything particularly wrong with the specific item chosen off of the todo list- it'd be nice to be able to tell what objects exist in a tablespace even if they're in other databases. The todo entry even talks about why it's difficult to do and what the expected way to go about doing it is (that is, connect to each database that has objects in the tablespace and query it to find out what's in the tablespace). Craig's suggestion is an interesting alternative way though and I'm not sure that it'd be all that bad, but it would be limited to catalog tables. If we'd extend the system to allow transparent usage of postgres_fdw to access other databases which are part of the same cluster, then this could end up being much simpler (eg: select * from otherdatabase.pg_catalog.pg_class ...). Thanks! Stephen signature.asc Description: PGP signature
TODO list (was Re: Contributing with code)
Robert Haas writes: > Also, let's delete the TODO list. People keep using it as a source of > project ideas, and that's bad. If we're not going to maintain/curate it properly, I agree it's not worth keeping it around. But I'd rather see somebody put some effort into it ... regards, tom lane
