Re: Policy should be simple (was "Allowed Champions on podlings")
On Tue, Mar 22, 2016 at 9:35 PM Marvin Humphrey wrote: > On Tue, Mar 22, 2016 at 1:56 PM, Roman Shaposhnik > wrote: > > On Tue, Mar 22, 2016 at 1:18 PM, Julian Hyde > wrote: > >> If you want to simplify policy, get rid of the champion. > >> Or rather, reduce the champion’s role to nominating a project for > incubation. > >> Once the project has entered incubation, the champion’s role ends. > > > > +1000 to the above. With all of my experience getting projects through > the > > incubation I couldn't find a single instance where a Champion role would > > prove to be really valuable ON TOP of active mentor(s). > > +1, I would be in favor of this. > > Let's make sure that we consider Julian's proposal separately from John's. > I'm assuming you mean here my proposed edit to who can champion a podling, right? > > The expansion of Champion role dates from 2011-2012, and I think it's fair > to > characterize the initiative as an attempt to deal with Mentor attrition: > > https://s.apache.org/XcKn > > Yes, that's the idea - basically make sure the Champion makes sure the > podling is reporting consistently and its mentors are present. > > Thanks largely to Roman's efforts to get Mentor signoff happening > consistently, we've addressed Mentor attrition a different way. > > 1. Mentors must sign off on podling reports, or the podling report is > removed from the monthly Board report. > 2. The Report Manager tracks missing podling reports. > 3. When a podling has not reported for 2 months in a row, the Mentors are > contacted via private@incubator. > 4. If no Mentors are available to resolve the issue, then the issue is > escalated to a public search for new Mentors. > > Back when expanding the Champion role was considered, there was stiff > resistance to Mentor signoff, so our current approach was not feasible. > However, it seems to me that the expanded Champion role is now obsolete and > that therefore Julian's proposal is sound. > I knew I wasn't making stuff up (again). I suppose what happened was that people ratified Roman's changes, but then the site was never updated. > > Here is a patch: > > https://paste.apache.org/Ersh > > If no one formally lodges a -1, I will claim lazy consensus and apply this > patch no sooner than a week from now, after discussion has died down. > One minor question - do we eliminate the champion attribute of existing podlings? Do we simply stop recording champion? How do we ensure that the champion is actually doing stuff instead of someone using that champion's name in vain? > > Marvin Humphrey > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > >
Re: Policy should be simple (was "Allowed Champions on podlings")
On Tue, Mar 22, 2016 at 1:56 PM, Roman Shaposhnik wrote: > On Tue, Mar 22, 2016 at 1:18 PM, Julian Hyde wrote: >> If you want to simplify policy, get rid of the champion. >> Or rather, reduce the champion’s role to nominating a project for incubation. >> Once the project has entered incubation, the champion’s role ends. > > +1000 to the above. With all of my experience getting projects through the > incubation I couldn't find a single instance where a Champion role would > prove to be really valuable ON TOP of active mentor(s). +1, I would be in favor of this. Let's make sure that we consider Julian's proposal separately from John's. The expansion of Champion role dates from 2011-2012, and I think it's fair to characterize the initiative as an attempt to deal with Mentor attrition: https://s.apache.org/XcKn Yes, that's the idea - basically make sure the Champion makes sure the podling is reporting consistently and its mentors are present. Thanks largely to Roman's efforts to get Mentor signoff happening consistently, we've addressed Mentor attrition a different way. 1. Mentors must sign off on podling reports, or the podling report is removed from the monthly Board report. 2. The Report Manager tracks missing podling reports. 3. When a podling has not reported for 2 months in a row, the Mentors are contacted via private@incubator. 4. If no Mentors are available to resolve the issue, then the issue is escalated to a public search for new Mentors. Back when expanding the Champion role was considered, there was stiff resistance to Mentor signoff, so our current approach was not feasible. However, it seems to me that the expanded Champion role is now obsolete and that therefore Julian's proposal is sound. Here is a patch: https://paste.apache.org/Ersh If no one formally lodges a -1, I will claim lazy consensus and apply this patch no sooner than a week from now, after discussion has died down. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Policy should be simple (was "Allowed Champions on podlings")
Roman Shaposhnik wrote on 3/22/16 4:56 PM: > On Tue, Mar 22, 2016 at 1:18 PM, Julian Hyde wrote: >> If you want to simplify policy, get rid of the champion. >> Or rather, reduce the champion’s role to nominating a project for incubation. >> Once the project has entered incubation, the champion’s role ends. > > +1000 to the above. With all of my experience getting projects through the > incubation I couldn't find a single instance where a Champion role would > prove to be really valuable ON TOP of active mentor(s). With the most important point in this entire thread being "active mentors". No parenthesis around the (s), I mean multiple mentors who are truly engaged with the podling, actively assisting, and ensuring that they review the quarterly podling reports going to the IPMC (and then to the board). If having some ASF Member or the like get the title "Champion" is shown to help ensure we have active mentors, then we need to keep it. If that's not true, then I agree: needless complication. A secondary concern (from reading board reports over the years) is ensuring that enough mentors on each podling *truly* get the Apache Way, so that we know a podling going towards graduation really is ready, and that we have multiple inputs to that evaluation. Overall, Incubator continues to improve, both with activity and clarity of process, which is awesome. 8-) - Shane - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Accept Quickstep into the Apache Incubator
+1 (binding) > On Mar 22, 2016, at 3:00 PM, Chris Douglas wrote: > > +1 (binding) -C > > On Tue, Mar 22, 2016 at 2:01 PM, Roman Shaposhnik wrote: >> Hi! >> >> Quickstep proposal was made available for discussion last week >>https://wiki.apache.org/incubator/QuickstepProposal >> and the feedback so far seems to be positive. >> >> Please vote to accept Quickstep into the Apache Incubator. >> The vote will be open until Mon 3/28 noon PST. >> >> [ ] +1 Accept Quickstep into the Apache Incubator >> [ ] +0 Abstain >> [ ] -1 Don't accept Quickstep into the Apache Incubator because ... >> >> == Abstract == >> >> Quickstep is a high-performance database engine. It is designed to (1) >> convert data to insights at bare-metal speed, (2) support multiple >> query surfaces including SQL (the first (and current) version only >> supports SQL, and (3) deliver bare-metal performance on any hardware >> (including running on a laptop, running on a high-end (single node) >> server, and running on a distributed cluster). Since its inception, >> the project has been planned to deliver a high-performance single node >> system first, followed by a distributed system. >> >> Quickstep is composed of several different modules that handle >> different concerns of a database system. The main modules are: >> * Utility - Reusable general-purpose code that is used by many other >> modules. >> * Threading - Provides a cross-platform abstraction for threads and >> synchronization primitives that abstract the underlying OS threading >> features. >> * Types - The core type system used across all of Quickstep. Handles >> details of how SQL types are stored, parsed, serialized & >> deserialized, and converted. Also includes basic containers for typed >> values (tuples and column-vectors) and low-level operations that apply >> to typed values (e.g. basic arithmetic and comparisons). >> * Catalog - Tracks database schema as well as physical storage >> information for relations (e.g. which physical blocks store a >> relation's data, and any physical partitioning and placement >> information). >> * Storage - Physically stores relational data in self-contained, >> self-describing blocks, both in-memory and on persistent storage (disk >> or a distributed filesystem). Also includes some heavyweight run-time >> data structures used in query processing (e.g. hash tables for join >> and aggregation). Includes a buffer manager component for managing >> memory use and a file manager component that handles data persistence. >> * Compression - Implements ordered dictionary compression. Several >> storage formats in the Storage module are capable of storing >> compressed column data and evaluating some expressions directly on >> compressed data without decompressing. The common code supporting >> compression is in this module. >> * Expressions - Builds on the simple operations provided by the >> Types module to support arbitrarily complex expressions over data, >> including scalar expressions, predicates, and aggregate functions with >> and without grouping. >> * Relational Operators - This module provides the building blocks >> for queries in Quickstep. A query is represented as a directed acyclic >> graph of relational operators, each of which is responsible for >> applying some relational-algebraic operation(s) to transform its >> input. Operators generate individual self-contained "work orders" that >> can be executed independently. Most operators are parallelism-friendly >> and generate one work-order per storage block of input. >> * Query Execution - Handles the actual scheduling and execution of >> work from a query at runtime. The central class is the Foreman, an >> independent thread with a global view of the query plan and progress. >> The Foreman dispatches work-orders to stateless Worker threads and >> monitors their progress, and also coordinates streaming of partial >> results between producers and consumers in a query plan DAG to >> maximize parallelism. This module also includes the QueryContext >> class, which holds global shared state for an individual query and is >> designed to support easy serialization/deserialization for distributed >> execution. >> * Parser - A simple SQL lexer and parser that parses SQL syntax into >> an abstract syntax tree for consumption by the Query Optimizer. >> * Query Optimizer - Takes the abstract syntax tree generated by the >> parser and transforms it into a runable query-plan DAG for the Query >> Execution module. The Query Optimizer is responsible for resolving >> references to relations and attributes in the query, checking it for >> semantic correctness, and applying optimizations (e.g. filter >> pushdown, column pruning, join ordering) as part of the transformation >> process. >> * Command-Line Interface - An interactive SQL shell interface to Quickstep. >> >> Quickstep is implemented in C++ and does not require many external >> libraries to run. Quickstep is currently an o
Re: [VOTE] Release Apache Mynewt 0.8.0-b2-incubating-rc3
+1 binding. Personally tested release on Arduino Zero and Simulated environments. Have seen it working on NRF51, NRF52 environments. Sterling On 3/22/16 2:50 PM, Justin Mclean wrote: Hi, +1 binding Carried over from the dev list. Did all of my usual checks. Note if you’re compiling on OS X you may need to make a couple of minor mods to build.sh. Thanks, Justin - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Accept Quickstep into the Apache Incubator
+1 (binding) -C On Tue, Mar 22, 2016 at 2:01 PM, Roman Shaposhnik wrote: > Hi! > > Quickstep proposal was made available for discussion last week > https://wiki.apache.org/incubator/QuickstepProposal > and the feedback so far seems to be positive. > > Please vote to accept Quickstep into the Apache Incubator. > The vote will be open until Mon 3/28 noon PST. > > [ ] +1 Accept Quickstep into the Apache Incubator > [ ] +0 Abstain > [ ] -1 Don't accept Quickstep into the Apache Incubator because ... > > == Abstract == > > Quickstep is a high-performance database engine. It is designed to (1) > convert data to insights at bare-metal speed, (2) support multiple > query surfaces including SQL (the first (and current) version only > supports SQL, and (3) deliver bare-metal performance on any hardware > (including running on a laptop, running on a high-end (single node) > server, and running on a distributed cluster). Since its inception, > the project has been planned to deliver a high-performance single node > system first, followed by a distributed system. > > Quickstep is composed of several different modules that handle > different concerns of a database system. The main modules are: > * Utility - Reusable general-purpose code that is used by many other > modules. > * Threading - Provides a cross-platform abstraction for threads and > synchronization primitives that abstract the underlying OS threading > features. > * Types - The core type system used across all of Quickstep. Handles > details of how SQL types are stored, parsed, serialized & > deserialized, and converted. Also includes basic containers for typed > values (tuples and column-vectors) and low-level operations that apply > to typed values (e.g. basic arithmetic and comparisons). > * Catalog - Tracks database schema as well as physical storage > information for relations (e.g. which physical blocks store a > relation's data, and any physical partitioning and placement > information). > * Storage - Physically stores relational data in self-contained, > self-describing blocks, both in-memory and on persistent storage (disk > or a distributed filesystem). Also includes some heavyweight run-time > data structures used in query processing (e.g. hash tables for join > and aggregation). Includes a buffer manager component for managing > memory use and a file manager component that handles data persistence. > * Compression - Implements ordered dictionary compression. Several > storage formats in the Storage module are capable of storing > compressed column data and evaluating some expressions directly on > compressed data without decompressing. The common code supporting > compression is in this module. > * Expressions - Builds on the simple operations provided by the > Types module to support arbitrarily complex expressions over data, > including scalar expressions, predicates, and aggregate functions with > and without grouping. > * Relational Operators - This module provides the building blocks > for queries in Quickstep. A query is represented as a directed acyclic > graph of relational operators, each of which is responsible for > applying some relational-algebraic operation(s) to transform its > input. Operators generate individual self-contained "work orders" that > can be executed independently. Most operators are parallelism-friendly > and generate one work-order per storage block of input. > * Query Execution - Handles the actual scheduling and execution of > work from a query at runtime. The central class is the Foreman, an > independent thread with a global view of the query plan and progress. > The Foreman dispatches work-orders to stateless Worker threads and > monitors their progress, and also coordinates streaming of partial > results between producers and consumers in a query plan DAG to > maximize parallelism. This module also includes the QueryContext > class, which holds global shared state for an individual query and is > designed to support easy serialization/deserialization for distributed > execution. > * Parser - A simple SQL lexer and parser that parses SQL syntax into > an abstract syntax tree for consumption by the Query Optimizer. > * Query Optimizer - Takes the abstract syntax tree generated by the > parser and transforms it into a runable query-plan DAG for the Query > Execution module. The Query Optimizer is responsible for resolving > references to relations and attributes in the query, checking it for > semantic correctness, and applying optimizations (e.g. filter > pushdown, column pruning, join ordering) as part of the transformation > process. > * Command-Line Interface - An interactive SQL shell interface to Quickstep. > > Quickstep is implemented in C++ and does not require many external > libraries to run. Quickstep is currently an open source project > licensed under the Apache License Version 2.0 and governed by a group > of engineers at Pivotal. > > Quickstep began in 2011 as a rese
Re: [VOTE] Release Apache Mynewt 0.8.0-b2-incubating-rc3
Hi, +1 binding Carried over from the dev list. Did all of my usual checks. Note if you’re compiling on OS X you may need to make a couple of minor mods to build.sh. Thanks, Justin - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [VOTE] Accept Quickstep into the Apache Incubator
+1 binding On Tue, Mar 22, 2016 at 9:01 PM, Roman Shaposhnik wrote: > Hi! > > Quickstep proposal was made available for discussion last week > https://wiki.apache.org/incubator/QuickstepProposal > and the feedback so far seems to be positive. > > Please vote to accept Quickstep into the Apache Incubator. > The vote will be open until Mon 3/28 noon PST. > > [ ] +1 Accept Quickstep into the Apache Incubator > [ ] +0 Abstain > [ ] -1 Don't accept Quickstep into the Apache Incubator because ... > > == Abstract == > > Quickstep is a high-performance database engine. It is designed to (1) > convert data to insights at bare-metal speed, (2) support multiple > query surfaces including SQL (the first (and current) version only > supports SQL, and (3) deliver bare-metal performance on any hardware > (including running on a laptop, running on a high-end (single node) > server, and running on a distributed cluster). Since its inception, > the project has been planned to deliver a high-performance single node > system first, followed by a distributed system. > > Quickstep is composed of several different modules that handle > different concerns of a database system. The main modules are: > * Utility - Reusable general-purpose code that is used by many other > modules. > * Threading - Provides a cross-platform abstraction for threads and > synchronization primitives that abstract the underlying OS threading > features. > * Types - The core type system used across all of Quickstep. Handles > details of how SQL types are stored, parsed, serialized & > deserialized, and converted. Also includes basic containers for typed > values (tuples and column-vectors) and low-level operations that apply > to typed values (e.g. basic arithmetic and comparisons). > * Catalog - Tracks database schema as well as physical storage > information for relations (e.g. which physical blocks store a > relation's data, and any physical partitioning and placement > information). > * Storage - Physically stores relational data in self-contained, > self-describing blocks, both in-memory and on persistent storage (disk > or a distributed filesystem). Also includes some heavyweight run-time > data structures used in query processing (e.g. hash tables for join > and aggregation). Includes a buffer manager component for managing > memory use and a file manager component that handles data persistence. > * Compression - Implements ordered dictionary compression. Several > storage formats in the Storage module are capable of storing > compressed column data and evaluating some expressions directly on > compressed data without decompressing. The common code supporting > compression is in this module. > * Expressions - Builds on the simple operations provided by the > Types module to support arbitrarily complex expressions over data, > including scalar expressions, predicates, and aggregate functions with > and without grouping. > * Relational Operators - This module provides the building blocks > for queries in Quickstep. A query is represented as a directed acyclic > graph of relational operators, each of which is responsible for > applying some relational-algebraic operation(s) to transform its > input. Operators generate individual self-contained "work orders" that > can be executed independently. Most operators are parallelism-friendly > and generate one work-order per storage block of input. > * Query Execution - Handles the actual scheduling and execution of > work from a query at runtime. The central class is the Foreman, an > independent thread with a global view of the query plan and progress. > The Foreman dispatches work-orders to stateless Worker threads and > monitors their progress, and also coordinates streaming of partial > results between producers and consumers in a query plan DAG to > maximize parallelism. This module also includes the QueryContext > class, which holds global shared state for an individual query and is > designed to support easy serialization/deserialization for distributed > execution. > * Parser - A simple SQL lexer and parser that parses SQL syntax into > an abstract syntax tree for consumption by the Query Optimizer. > * Query Optimizer - Takes the abstract syntax tree generated by the > parser and transforms it into a runable query-plan DAG for the Query > Execution module. The Query Optimizer is responsible for resolving > references to relations and attributes in the query, checking it for > semantic correctness, and applying optimizations (e.g. filter > pushdown, column pruning, join ordering) as part of the transformation > process. > * Command-Line Interface - An interactive SQL shell interface to > Quickstep. > > Quickstep is implemented in C++ and does not require many external > libraries to run. Quickstep is currently an open source project > licensed under the Apache License Version 2.0 and governed by a group > of engineers at Pivotal. > > Quickstep began in 2011 as a researc
Re: [VOTE] Accept Quickstep into the Apache Incubator
+1 (binding) --Chris Nauroth On 3/22/16, 2:01 PM, "Roman Shaposhnik" wrote: >Hi! > >Quickstep proposal was made available for discussion last week >https://wiki.apache.org/incubator/QuickstepProposal >and the feedback so far seems to be positive. > >Please vote to accept Quickstep into the Apache Incubator. >The vote will be open until Mon 3/28 noon PST. > >[ ] +1 Accept Quickstep into the Apache Incubator >[ ] +0 Abstain >[ ] -1 Don't accept Quickstep into the Apache Incubator because ... > >== Abstract == > >Quickstep is a high-performance database engine. It is designed to (1) >convert data to insights at bare-metal speed, (2) support multiple >query surfaces including SQL (the first (and current) version only >supports SQL, and (3) deliver bare-metal performance on any hardware >(including running on a laptop, running on a high-end (single node) >server, and running on a distributed cluster). Since its inception, >the project has been planned to deliver a high-performance single node >system first, followed by a distributed system. > >Quickstep is composed of several different modules that handle >different concerns of a database system. The main modules are: > * Utility - Reusable general-purpose code that is used by many other >modules. > * Threading - Provides a cross-platform abstraction for threads and >synchronization primitives that abstract the underlying OS threading >features. > * Types - The core type system used across all of Quickstep. Handles >details of how SQL types are stored, parsed, serialized & >deserialized, and converted. Also includes basic containers for typed >values (tuples and column-vectors) and low-level operations that apply >to typed values (e.g. basic arithmetic and comparisons). > * Catalog - Tracks database schema as well as physical storage >information for relations (e.g. which physical blocks store a >relation's data, and any physical partitioning and placement >information). > * Storage - Physically stores relational data in self-contained, >self-describing blocks, both in-memory and on persistent storage (disk >or a distributed filesystem). Also includes some heavyweight run-time >data structures used in query processing (e.g. hash tables for join >and aggregation). Includes a buffer manager component for managing >memory use and a file manager component that handles data persistence. > * Compression - Implements ordered dictionary compression. Several >storage formats in the Storage module are capable of storing >compressed column data and evaluating some expressions directly on >compressed data without decompressing. The common code supporting >compression is in this module. > * Expressions - Builds on the simple operations provided by the >Types module to support arbitrarily complex expressions over data, >including scalar expressions, predicates, and aggregate functions with >and without grouping. > * Relational Operators - This module provides the building blocks >for queries in Quickstep. A query is represented as a directed acyclic >graph of relational operators, each of which is responsible for >applying some relational-algebraic operation(s) to transform its >input. Operators generate individual self-contained "work orders" that >can be executed independently. Most operators are parallelism-friendly >and generate one work-order per storage block of input. > * Query Execution - Handles the actual scheduling and execution of >work from a query at runtime. The central class is the Foreman, an >independent thread with a global view of the query plan and progress. >The Foreman dispatches work-orders to stateless Worker threads and >monitors their progress, and also coordinates streaming of partial >results between producers and consumers in a query plan DAG to >maximize parallelism. This module also includes the QueryContext >class, which holds global shared state for an individual query and is >designed to support easy serialization/deserialization for distributed >execution. > * Parser - A simple SQL lexer and parser that parses SQL syntax into >an abstract syntax tree for consumption by the Query Optimizer. > * Query Optimizer - Takes the abstract syntax tree generated by the >parser and transforms it into a runable query-plan DAG for the Query >Execution module. The Query Optimizer is responsible for resolving >references to relations and attributes in the query, checking it for >semantic correctness, and applying optimizations (e.g. filter >pushdown, column pruning, join ordering) as part of the transformation >process. > * Command-Line Interface - An interactive SQL shell interface to >Quickstep. > >Quickstep is implemented in C++ and does not require many external >libraries to run. Quickstep is currently an open source project >licensed under the Apache License Version 2.0 and governed by a group >of engineers at Pivotal. > >Quickstep began in 2011 as a research project in the Computer Sciences >Department at the University of Wi
Re: [VOTE] Accept Quickstep into the Apache Incubator
+1 (binding) Regards JB On 03/22/2016 10:01 PM, Roman Shaposhnik wrote: Hi! Quickstep proposal was made available for discussion last week https://wiki.apache.org/incubator/QuickstepProposal and the feedback so far seems to be positive. Please vote to accept Quickstep into the Apache Incubator. The vote will be open until Mon 3/28 noon PST. [ ] +1 Accept Quickstep into the Apache Incubator [ ] +0 Abstain [ ] -1 Don't accept Quickstep into the Apache Incubator because ... == Abstract == Quickstep is a high-performance database engine. It is designed to (1) convert data to insights at bare-metal speed, (2) support multiple query surfaces including SQL (the first (and current) version only supports SQL, and (3) deliver bare-metal performance on any hardware (including running on a laptop, running on a high-end (single node) server, and running on a distributed cluster). Since its inception, the project has been planned to deliver a high-performance single node system first, followed by a distributed system. Quickstep is composed of several different modules that handle different concerns of a database system. The main modules are: * Utility - Reusable general-purpose code that is used by many other modules. * Threading - Provides a cross-platform abstraction for threads and synchronization primitives that abstract the underlying OS threading features. * Types - The core type system used across all of Quickstep. Handles details of how SQL types are stored, parsed, serialized & deserialized, and converted. Also includes basic containers for typed values (tuples and column-vectors) and low-level operations that apply to typed values (e.g. basic arithmetic and comparisons). * Catalog - Tracks database schema as well as physical storage information for relations (e.g. which physical blocks store a relation's data, and any physical partitioning and placement information). * Storage - Physically stores relational data in self-contained, self-describing blocks, both in-memory and on persistent storage (disk or a distributed filesystem). Also includes some heavyweight run-time data structures used in query processing (e.g. hash tables for join and aggregation). Includes a buffer manager component for managing memory use and a file manager component that handles data persistence. * Compression - Implements ordered dictionary compression. Several storage formats in the Storage module are capable of storing compressed column data and evaluating some expressions directly on compressed data without decompressing. The common code supporting compression is in this module. * Expressions - Builds on the simple operations provided by the Types module to support arbitrarily complex expressions over data, including scalar expressions, predicates, and aggregate functions with and without grouping. * Relational Operators - This module provides the building blocks for queries in Quickstep. A query is represented as a directed acyclic graph of relational operators, each of which is responsible for applying some relational-algebraic operation(s) to transform its input. Operators generate individual self-contained "work orders" that can be executed independently. Most operators are parallelism-friendly and generate one work-order per storage block of input. * Query Execution - Handles the actual scheduling and execution of work from a query at runtime. The central class is the Foreman, an independent thread with a global view of the query plan and progress. The Foreman dispatches work-orders to stateless Worker threads and monitors their progress, and also coordinates streaming of partial results between producers and consumers in a query plan DAG to maximize parallelism. This module also includes the QueryContext class, which holds global shared state for an individual query and is designed to support easy serialization/deserialization for distributed execution. * Parser - A simple SQL lexer and parser that parses SQL syntax into an abstract syntax tree for consumption by the Query Optimizer. * Query Optimizer - Takes the abstract syntax tree generated by the parser and transforms it into a runable query-plan DAG for the Query Execution module. The Query Optimizer is responsible for resolving references to relations and attributes in the query, checking it for semantic correctness, and applying optimizations (e.g. filter pushdown, column pruning, join ordering) as part of the transformation process. * Command-Line Interface - An interactive SQL shell interface to Quickstep. Quickstep is implemented in C++ and does not require many external libraries to run. Quickstep is currently an open source project licensed under the Apache License Version 2.0 and governed by a group of engineers at Pivotal. Quickstep began in 2011 as a research project in the Computer Sciences Department at the University of Wisconsin https://quickstep.cs.wisc.edu/ and the copyrights underlying the project was
[VOTE] Accept Quickstep into the Apache Incubator
Hi! Quickstep proposal was made available for discussion last week https://wiki.apache.org/incubator/QuickstepProposal and the feedback so far seems to be positive. Please vote to accept Quickstep into the Apache Incubator. The vote will be open until Mon 3/28 noon PST. [ ] +1 Accept Quickstep into the Apache Incubator [ ] +0 Abstain [ ] -1 Don't accept Quickstep into the Apache Incubator because ... == Abstract == Quickstep is a high-performance database engine. It is designed to (1) convert data to insights at bare-metal speed, (2) support multiple query surfaces including SQL (the first (and current) version only supports SQL, and (3) deliver bare-metal performance on any hardware (including running on a laptop, running on a high-end (single node) server, and running on a distributed cluster). Since its inception, the project has been planned to deliver a high-performance single node system first, followed by a distributed system. Quickstep is composed of several different modules that handle different concerns of a database system. The main modules are: * Utility - Reusable general-purpose code that is used by many other modules. * Threading - Provides a cross-platform abstraction for threads and synchronization primitives that abstract the underlying OS threading features. * Types - The core type system used across all of Quickstep. Handles details of how SQL types are stored, parsed, serialized & deserialized, and converted. Also includes basic containers for typed values (tuples and column-vectors) and low-level operations that apply to typed values (e.g. basic arithmetic and comparisons). * Catalog - Tracks database schema as well as physical storage information for relations (e.g. which physical blocks store a relation's data, and any physical partitioning and placement information). * Storage - Physically stores relational data in self-contained, self-describing blocks, both in-memory and on persistent storage (disk or a distributed filesystem). Also includes some heavyweight run-time data structures used in query processing (e.g. hash tables for join and aggregation). Includes a buffer manager component for managing memory use and a file manager component that handles data persistence. * Compression - Implements ordered dictionary compression. Several storage formats in the Storage module are capable of storing compressed column data and evaluating some expressions directly on compressed data without decompressing. The common code supporting compression is in this module. * Expressions - Builds on the simple operations provided by the Types module to support arbitrarily complex expressions over data, including scalar expressions, predicates, and aggregate functions with and without grouping. * Relational Operators - This module provides the building blocks for queries in Quickstep. A query is represented as a directed acyclic graph of relational operators, each of which is responsible for applying some relational-algebraic operation(s) to transform its input. Operators generate individual self-contained "work orders" that can be executed independently. Most operators are parallelism-friendly and generate one work-order per storage block of input. * Query Execution - Handles the actual scheduling and execution of work from a query at runtime. The central class is the Foreman, an independent thread with a global view of the query plan and progress. The Foreman dispatches work-orders to stateless Worker threads and monitors their progress, and also coordinates streaming of partial results between producers and consumers in a query plan DAG to maximize parallelism. This module also includes the QueryContext class, which holds global shared state for an individual query and is designed to support easy serialization/deserialization for distributed execution. * Parser - A simple SQL lexer and parser that parses SQL syntax into an abstract syntax tree for consumption by the Query Optimizer. * Query Optimizer - Takes the abstract syntax tree generated by the parser and transforms it into a runable query-plan DAG for the Query Execution module. The Query Optimizer is responsible for resolving references to relations and attributes in the query, checking it for semantic correctness, and applying optimizations (e.g. filter pushdown, column pruning, join ordering) as part of the transformation process. * Command-Line Interface - An interactive SQL shell interface to Quickstep. Quickstep is implemented in C++ and does not require many external libraries to run. Quickstep is currently an open source project licensed under the Apache License Version 2.0 and governed by a group of engineers at Pivotal. Quickstep began in 2011 as a research project in the Computer Sciences Department at the University of Wisconsin https://quickstep.cs.wisc.edu/ and the copyrights underlying the project was transferred to a company called Quickstep Technologies, which was acquired by Pivotal in
Re: Policy should be simple (was "Allowed Champions on podlings")
On Tue, Mar 22, 2016 at 1:18 PM, Julian Hyde wrote: > If you want to simplify policy, get rid of the champion. > Or rather, reduce the champion’s role to nominating a project for incubation. > Once the project has entered incubation, the champion’s role ends. +1000 to the above. With all of my experience getting projects through the incubation I couldn't find a single instance where a Champion role would prove to be really valuable ON TOP of active mentor(s). Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
[VOTE] Release Apache Mynewt 0.8.0-b2-incubating-rc3
Hello, The Apache Mynewt Incubator PPMC has approved a proposal to release Release Apache Mynewt 0.8.0-b2-incubating-rc3. We now kindly request that the Incubator PMC members review and vote on this incubator release. Apache Mynewt is a community-driven, permissively licensed open source initiative for constrained, embedded applications. Mynewt provides a real-time operating system, flash file system, network stacks, and support utilities for real-world embedded systems. For full release notes, please visit the Apache Mynewt Wiki: https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes For information about bugs fixed in beta 2, please view the Issue Tracker Summary: https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319221&version=12334805 This is the first source release of the "New" Newt. More information on the genesis of Newt, and a description can be found at: http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201603.mbox/%3C56E21C13.9050303%40apache.org%3E The full mynewt test suite for this release was executed via "newt test all" with no failures. This testing was performed on the following platforms: * OS X 10.10.5 * Linux 3.19.0 (Ubuntu 14.10) In addition, all other currently-supported platforms were smoke tested. The release candidate to be voted on is available at: https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b2-incubating/rc3/ [VOTE] thread: http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201603.mbox/%3C20160319014649.GC26120%40iori.nightmare-heaven.no-ip.biz%3E [RESULT][VOTE] thread: http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201603.mbox/%3C20160322195251.GJ26120%40iori.nightmare-heaven.no-ip.biz%3E [DISCUSS] thread: http://mail-archives.apache.org/mod_mbox/incubator-mynewt-dev/201603.mbox/%3C20160319014750.GD26120%40iori.nightmare-heaven.no-ip.biz%3E The release candidate to be voted on is available at: https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b2-incubating/rc3/ The commits under consideration are as follows: blinky (formerly tadpole): repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-blinky.git commit: 82f09fa2cc4c67fb30165ef6be0291aa76e6213e core (formerly larva): repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-core.git commit: 2bec33d4dba12094f32ca560410b8671c0ce1d22 newt: repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt.git commit: 58053579210a3541916d3e799b22cc81bc473e47 In addition, the following newt convenience binaries are available: linux: https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b2-incubating/rc3/apache-newt-bin-linux-0.8.0-b2-incubating.tgz osx: https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-0.8.0-b2-incubating/rc3/apache-newt-bin-osx-0.8.0-b2-incubating.tgz The release candidate is signed with a GPG key available at: https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS The vote is open for at least 72 hours. [ ] +1 Release this package [ ] 0 I don't feel strongly about it, but don't object [ ] -1 Do not release this package because... Thanks, Chris - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: The Apache Way incubator page
On Tue, 22 Mar 2016, Christopher wrote: The page at http://incubator.apache.org/learn/theapacheway.html needs to be updated to reflect the new location of the ComDev mailing lists. It is no longer commun...@apache.org (and community-subscr...@apache.org), but is instead d...@community.apache.org (and dev-subscr...@community.apache.org). Thanks, updated now! FYI, you can use the CMS to quickly generate patches for the site, and even commit + publish if you have karma. See http://www.apache.org/dev/cmsref.html for details of the bookmarklet that'll let you do so easily :) Nick - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [DISCUSS] Quickstep incubation proposal
No, absolutely my comment wasn't supposed to have any insinuation about whether the project should get incubated or not from a proposal perspective. It was just a round about way of saying, I like the proposal, its fresh, looks sane and is something that's a bit different so it gets a +1 from me. On Tue, Mar 22, 2016 at 7:09 PM, Konstantin Boudnik wrote: > That's a fair statement. In general, however, it isn't a concern of the > Incubator if a proposed podling have some sort of resemblance with some > other > software out there. IINM, no one was rejected because they want to develop > yet > another web-application server or something like this. > > Cos > > On Tue, Mar 22, 2016 at 06:44PM, Tom Barber wrote: > > I actually have an opinion! > > > > I saw yet another database engine land and my heart sank > > > > Then I did some digging into quickstep and realised it was more of a > > "traditional" database that might take on the likes of Exasol etc rather > > than plugging more SQL into NOSQL etc(from what I gather) and I am happy > to > > see it pitched. > > > > Tom > > > > On Tue, Mar 22, 2016 at 6:41 PM, Konstantin Boudnik > wrote: > > > > > It's been a week since this thread started and surprisingly there > isn't any > > > reaction so far. Is it safe to assume the silent consensus has been > > > reached? > > > > > > Cos > > > > > > On Tue, Mar 15, 2016 at 04:52PM, Roman Shaposhnik wrote: > > > > Hi! > > > > > > > > It is my pleasure to present the proposal to incubate the Quickstep > > > project > > > > at the Apache Software Foundation. Quickstep is a high-performance > > > > next generation, database engine available under Apache License 2.0. > > > > > > > > The text of the proposal is included below and is also available at > > > >https://wiki.apache.org/incubator/QuickstepProposal > > > > > > > > Thanks, > > > > Roman. > > > > > > > > == Abstract == > > > > > > > > Quickstep is a high-performance database engine. It is designed to > (1) > > > > convert data to insights at bare-metal speed, (2) support multiple > > > > query surfaces including SQL (the first (and current) version only > > > > supports SQL, and (3) deliver bare-metal performance on any hardware > > > > (including running on a laptop, running on a high-end (single node) > > > > server, and running on a distributed cluster). Since its inception, > > > > the project has been planned to deliver a high-performance single > node > > > > system first, followed by a distributed system. > > > > > > > > Quickstep is composed of several different modules that handle > > > > different concerns of a database system. The main modules are: > > > > * Utility - Reusable general-purpose code that is used by many > other > > > modules. > > > > * Threading - Provides a cross-platform abstraction for threads and > > > > synchronization primitives that abstract the underlying OS threading > > > > features. > > > > * Types - The core type system used across all of Quickstep. > Handles > > > > details of how SQL types are stored, parsed, serialized & > > > > deserialized, and converted. Also includes basic containers for typed > > > > values (tuples and column-vectors) and low-level operations that > apply > > > > to typed values (e.g. basic arithmetic and comparisons). > > > > * Catalog - Tracks database schema as well as physical storage > > > > information for relations (e.g. which physical blocks store a > > > > relation's data, and any physical partitioning and placement > > > > information). > > > > * Storage - Physically stores relational data in self-contained, > > > > self-describing blocks, both in-memory and on persistent storage > (disk > > > > or a distributed filesystem). Also includes some heavyweight run-time > > > > data structures used in query processing (e.g. hash tables for join > > > > and aggregation). Includes a buffer manager component for managing > > > > memory use and a file manager component that handles data > persistence. > > > > * Compression - Implements ordered dictionary compression. Several > > > > storage formats in the Storage module are capable of storing > > > > compressed column data and evaluating some expressions directly on > > > > compressed data without decompressing. The common code supporting > > > > compression is in this module. > > > > * Expressions - Builds on the simple operations provided by the > > > > Types module to support arbitrarily complex expressions over data, > > > > including scalar expressions, predicates, and aggregate functions > with > > > > and without grouping. > > > > * Relational Operators - This module provides the building blocks > > > > for queries in Quickstep. A query is represented as a directed > acyclic > > > > graph of relational operators, each of which is responsible for > > > > applying some relational-algebraic operation(s) to transform its > > > > input. Operators generate individual self-contained "work orders" > that > > > > can be executed inde
The Apache Way incubator page
The page at http://incubator.apache.org/learn/theapacheway.html needs to be updated to reflect the new location of the ComDev mailing lists. It is no longer commun...@apache.org (and community-subscr...@apache.org), but is instead d...@community.apache.org (and dev-subscr...@community.apache.org).
Re: Policy should be simple (was "Allowed Champions on podlings")
If you want to simplify policy, get rid of the champion. Or rather, reduce the champion’s role to nominating a project for incubation. Once the project has entered incubation, the champion’s role ends. While Calcite was in incubation no one (including the champion) had a clue what the role of the champion was. Was the champion implicitly on the PPMC? Did they have a binding vote? Were they implicitly a mentor? If we needed karma to do xyz, should we email the champion or the mentors? It would be simpler if there were just mentors. And of course the champion can become a mentor if he/she wants to. Julian > On Mar 22, 2016, at 11:39 AM, Craig Russell wrote: > >> >> On Mar 22, 2016, at 11:25 AM, John D. Ament wrote: >> >> On Tue, Mar 22, 2016 at 12:19 PM Marvin Humphrey >> wrote: >> >>> On Mon, Mar 21, 2016 at 9:57 AM, Craig Russell >>> wrote: There is a sorta technical reason for the Champion to be a member of the >>> PMC of the sponsor. I’d expect the Champion to subscribe to the private@ list and to have binding votes on podling releases. These both require PMC membership. The alternative is to create two different “exceptions” that would allow Champions to subscribe to private@ and to have binding release votes. >>> >>> These are legitimate concerns that would need to be dealt should such an >>> unlikely scenario arise. However, I don't think we need to carve >>> exceptions >>> into policy here -- other creative solutions are available, like voting the >>> Champion onto the Sponsor PMC. >>> >> >> Moreso, the champion is responsible for bringing the project into the ASF. >> The Champion holds no bearing on the project after that point. >> >> It sounds like what we're being pitched here is that the champion must stay >> on to mentor the project. That hasn't been followed too much. > > I did not think this thread was discussing the role of the Champion, just the > pre-requisites. > > http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Champion > > During incubation, the Champion: > > • Coordinates the creation and timely delivery of the podling's board > reports. > • Keeps an eye on the mentors' activity and takes action (ask for new > mentors, talk to the Incubator PMC) if they don't seem to provide enough > oversight or mentorship to the podling, > The podling can elect a new Champion at any time, and must notify the > Incubator PMC when that happens. > > The podling reports indicate who the current Champion is, and that > information is kept up to date in the Incubator PMC's records (currently > content/podlings.xml@champion). > > Craig >> >> >>> >>> And I'd like to take this opportunity to make a more general point: >>> >>> Policy should be simple. >>> >>> There are many reasons that policy should be simple, and I'm sure others >>> will >>> be happy to weigh in with their own favorites. But for me, this is the >>> most compelling: >>> >>> Complexity harms newcomers. >>> >>> Right now, Apache's rules are so complex that we are all in perpetual >>> violation. You can't even know what all the rules are! >>> >>> In such an environment, success is dependent, not on your own ability, but >>> on >>> securing alliances with powerful insiders who can help you bend or break >>> the rules. >>> >>> This state of affairs is not worthy of our core principles. Particularly >>> since the ASF does not exercise technical control over its projects, what >>> we >>> do here is not really that complicated. >>> >>> Apache, and the Incubator in particular, welcomes newcomers. It should be >>> possible for a newcomers to discover and follow our rules largely through >>> their own efforts. >>> >>> Of course a rejoinder to "Policy should be simple" is "As simple as >>> possible >>> and no simpler". But how close are we to "as simple as possible"? >>> >>> Marvin Humphrey >>> >>> - >>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >>> For additional commands, e-mail: general-h...@incubator.apache.org >>> >>> > > Craig L Russell > Architect > craig.russ...@oracle.com > P.S. A good JDO? O, Gasp! > > > > > > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: [DISCUSS] Quickstep incubation proposal
That's a fair statement. In general, however, it isn't a concern of the Incubator if a proposed podling have some sort of resemblance with some other software out there. IINM, no one was rejected because they want to develop yet another web-application server or something like this. Cos On Tue, Mar 22, 2016 at 06:44PM, Tom Barber wrote: > I actually have an opinion! > > I saw yet another database engine land and my heart sank > > Then I did some digging into quickstep and realised it was more of a > "traditional" database that might take on the likes of Exasol etc rather > than plugging more SQL into NOSQL etc(from what I gather) and I am happy to > see it pitched. > > Tom > > On Tue, Mar 22, 2016 at 6:41 PM, Konstantin Boudnik wrote: > > > It's been a week since this thread started and surprisingly there isn't any > > reaction so far. Is it safe to assume the silent consensus has been > > reached? > > > > Cos > > > > On Tue, Mar 15, 2016 at 04:52PM, Roman Shaposhnik wrote: > > > Hi! > > > > > > It is my pleasure to present the proposal to incubate the Quickstep > > project > > > at the Apache Software Foundation. Quickstep is a high-performance > > > next generation, database engine available under Apache License 2.0. > > > > > > The text of the proposal is included below and is also available at > > >https://wiki.apache.org/incubator/QuickstepProposal > > > > > > Thanks, > > > Roman. > > > > > > == Abstract == > > > > > > Quickstep is a high-performance database engine. It is designed to (1) > > > convert data to insights at bare-metal speed, (2) support multiple > > > query surfaces including SQL (the first (and current) version only > > > supports SQL, and (3) deliver bare-metal performance on any hardware > > > (including running on a laptop, running on a high-end (single node) > > > server, and running on a distributed cluster). Since its inception, > > > the project has been planned to deliver a high-performance single node > > > system first, followed by a distributed system. > > > > > > Quickstep is composed of several different modules that handle > > > different concerns of a database system. The main modules are: > > > * Utility - Reusable general-purpose code that is used by many other > > modules. > > > * Threading - Provides a cross-platform abstraction for threads and > > > synchronization primitives that abstract the underlying OS threading > > > features. > > > * Types - The core type system used across all of Quickstep. Handles > > > details of how SQL types are stored, parsed, serialized & > > > deserialized, and converted. Also includes basic containers for typed > > > values (tuples and column-vectors) and low-level operations that apply > > > to typed values (e.g. basic arithmetic and comparisons). > > > * Catalog - Tracks database schema as well as physical storage > > > information for relations (e.g. which physical blocks store a > > > relation's data, and any physical partitioning and placement > > > information). > > > * Storage - Physically stores relational data in self-contained, > > > self-describing blocks, both in-memory and on persistent storage (disk > > > or a distributed filesystem). Also includes some heavyweight run-time > > > data structures used in query processing (e.g. hash tables for join > > > and aggregation). Includes a buffer manager component for managing > > > memory use and a file manager component that handles data persistence. > > > * Compression - Implements ordered dictionary compression. Several > > > storage formats in the Storage module are capable of storing > > > compressed column data and evaluating some expressions directly on > > > compressed data without decompressing. The common code supporting > > > compression is in this module. > > > * Expressions - Builds on the simple operations provided by the > > > Types module to support arbitrarily complex expressions over data, > > > including scalar expressions, predicates, and aggregate functions with > > > and without grouping. > > > * Relational Operators - This module provides the building blocks > > > for queries in Quickstep. A query is represented as a directed acyclic > > > graph of relational operators, each of which is responsible for > > > applying some relational-algebraic operation(s) to transform its > > > input. Operators generate individual self-contained "work orders" that > > > can be executed independently. Most operators are parallelism-friendly > > > and generate one work-order per storage block of input. > > > * Query Execution - Handles the actual scheduling and execution of > > > work from a query at runtime. The central class is the Foreman, an > > > independent thread with a global view of the query plan and progress. > > > The Foreman dispatches work-orders to stateless Worker threads and > > > monitors their progress, and also coordinates streaming of partial > > > results between producers and consumers in a query plan DAG to > > > max
Re: [DISCUSS] Quickstep incubation proposal
I actually have an opinion! I saw yet another database engine land and my heart sank Then I did some digging into quickstep and realised it was more of a "traditional" database that might take on the likes of Exasol etc rather than plugging more SQL into NOSQL etc(from what I gather) and I am happy to see it pitched. Tom On Tue, Mar 22, 2016 at 6:41 PM, Konstantin Boudnik wrote: > It's been a week since this thread started and surprisingly there isn't any > reaction so far. Is it safe to assume the silent consensus has been > reached? > > Cos > > On Tue, Mar 15, 2016 at 04:52PM, Roman Shaposhnik wrote: > > Hi! > > > > It is my pleasure to present the proposal to incubate the Quickstep > project > > at the Apache Software Foundation. Quickstep is a high-performance > > next generation, database engine available under Apache License 2.0. > > > > The text of the proposal is included below and is also available at > >https://wiki.apache.org/incubator/QuickstepProposal > > > > Thanks, > > Roman. > > > > == Abstract == > > > > Quickstep is a high-performance database engine. It is designed to (1) > > convert data to insights at bare-metal speed, (2) support multiple > > query surfaces including SQL (the first (and current) version only > > supports SQL, and (3) deliver bare-metal performance on any hardware > > (including running on a laptop, running on a high-end (single node) > > server, and running on a distributed cluster). Since its inception, > > the project has been planned to deliver a high-performance single node > > system first, followed by a distributed system. > > > > Quickstep is composed of several different modules that handle > > different concerns of a database system. The main modules are: > > * Utility - Reusable general-purpose code that is used by many other > modules. > > * Threading - Provides a cross-platform abstraction for threads and > > synchronization primitives that abstract the underlying OS threading > > features. > > * Types - The core type system used across all of Quickstep. Handles > > details of how SQL types are stored, parsed, serialized & > > deserialized, and converted. Also includes basic containers for typed > > values (tuples and column-vectors) and low-level operations that apply > > to typed values (e.g. basic arithmetic and comparisons). > > * Catalog - Tracks database schema as well as physical storage > > information for relations (e.g. which physical blocks store a > > relation's data, and any physical partitioning and placement > > information). > > * Storage - Physically stores relational data in self-contained, > > self-describing blocks, both in-memory and on persistent storage (disk > > or a distributed filesystem). Also includes some heavyweight run-time > > data structures used in query processing (e.g. hash tables for join > > and aggregation). Includes a buffer manager component for managing > > memory use and a file manager component that handles data persistence. > > * Compression - Implements ordered dictionary compression. Several > > storage formats in the Storage module are capable of storing > > compressed column data and evaluating some expressions directly on > > compressed data without decompressing. The common code supporting > > compression is in this module. > > * Expressions - Builds on the simple operations provided by the > > Types module to support arbitrarily complex expressions over data, > > including scalar expressions, predicates, and aggregate functions with > > and without grouping. > > * Relational Operators - This module provides the building blocks > > for queries in Quickstep. A query is represented as a directed acyclic > > graph of relational operators, each of which is responsible for > > applying some relational-algebraic operation(s) to transform its > > input. Operators generate individual self-contained "work orders" that > > can be executed independently. Most operators are parallelism-friendly > > and generate one work-order per storage block of input. > > * Query Execution - Handles the actual scheduling and execution of > > work from a query at runtime. The central class is the Foreman, an > > independent thread with a global view of the query plan and progress. > > The Foreman dispatches work-orders to stateless Worker threads and > > monitors their progress, and also coordinates streaming of partial > > results between producers and consumers in a query plan DAG to > > maximize parallelism. This module also includes the QueryContext > > class, which holds global shared state for an individual query and is > > designed to support easy serialization/deserialization for distributed > > execution. > > * Parser - A simple SQL lexer and parser that parses SQL syntax into > > an abstract syntax tree for consumption by the Query Optimizer. > > * Query Optimizer - Takes the abstract syntax tree generated by the > > parser and transforms it into a runable query-plan DAG for the Query > >
Re: [DISCUSS] Quickstep incubation proposal
It's been a week since this thread started and surprisingly there isn't any reaction so far. Is it safe to assume the silent consensus has been reached? Cos On Tue, Mar 15, 2016 at 04:52PM, Roman Shaposhnik wrote: > Hi! > > It is my pleasure to present the proposal to incubate the Quickstep project > at the Apache Software Foundation. Quickstep is a high-performance > next generation, database engine available under Apache License 2.0. > > The text of the proposal is included below and is also available at >https://wiki.apache.org/incubator/QuickstepProposal > > Thanks, > Roman. > > == Abstract == > > Quickstep is a high-performance database engine. It is designed to (1) > convert data to insights at bare-metal speed, (2) support multiple > query surfaces including SQL (the first (and current) version only > supports SQL, and (3) deliver bare-metal performance on any hardware > (including running on a laptop, running on a high-end (single node) > server, and running on a distributed cluster). Since its inception, > the project has been planned to deliver a high-performance single node > system first, followed by a distributed system. > > Quickstep is composed of several different modules that handle > different concerns of a database system. The main modules are: > * Utility - Reusable general-purpose code that is used by many other > modules. > * Threading - Provides a cross-platform abstraction for threads and > synchronization primitives that abstract the underlying OS threading > features. > * Types - The core type system used across all of Quickstep. Handles > details of how SQL types are stored, parsed, serialized & > deserialized, and converted. Also includes basic containers for typed > values (tuples and column-vectors) and low-level operations that apply > to typed values (e.g. basic arithmetic and comparisons). > * Catalog - Tracks database schema as well as physical storage > information for relations (e.g. which physical blocks store a > relation's data, and any physical partitioning and placement > information). > * Storage - Physically stores relational data in self-contained, > self-describing blocks, both in-memory and on persistent storage (disk > or a distributed filesystem). Also includes some heavyweight run-time > data structures used in query processing (e.g. hash tables for join > and aggregation). Includes a buffer manager component for managing > memory use and a file manager component that handles data persistence. > * Compression - Implements ordered dictionary compression. Several > storage formats in the Storage module are capable of storing > compressed column data and evaluating some expressions directly on > compressed data without decompressing. The common code supporting > compression is in this module. > * Expressions - Builds on the simple operations provided by the > Types module to support arbitrarily complex expressions over data, > including scalar expressions, predicates, and aggregate functions with > and without grouping. > * Relational Operators - This module provides the building blocks > for queries in Quickstep. A query is represented as a directed acyclic > graph of relational operators, each of which is responsible for > applying some relational-algebraic operation(s) to transform its > input. Operators generate individual self-contained "work orders" that > can be executed independently. Most operators are parallelism-friendly > and generate one work-order per storage block of input. > * Query Execution - Handles the actual scheduling and execution of > work from a query at runtime. The central class is the Foreman, an > independent thread with a global view of the query plan and progress. > The Foreman dispatches work-orders to stateless Worker threads and > monitors their progress, and also coordinates streaming of partial > results between producers and consumers in a query plan DAG to > maximize parallelism. This module also includes the QueryContext > class, which holds global shared state for an individual query and is > designed to support easy serialization/deserialization for distributed > execution. > * Parser - A simple SQL lexer and parser that parses SQL syntax into > an abstract syntax tree for consumption by the Query Optimizer. > * Query Optimizer - Takes the abstract syntax tree generated by the > parser and transforms it into a runable query-plan DAG for the Query > Execution module. The Query Optimizer is responsible for resolving > references to relations and attributes in the query, checking it for > semantic correctness, and applying optimizations (e.g. filter > pushdown, column pruning, join ordering) as part of the transformation > process. > * Command-Line Interface - An interactive SQL shell interface to Quickstep. > > Quickstep is implemented in C++ and does not require many external > libraries to run. Quickstep is currently an open source project > licensed under the Apache License Version
Re: Policy should be simple (was "Allowed Champions on podlings")
> On Mar 22, 2016, at 11:25 AM, John D. Ament wrote: > > On Tue, Mar 22, 2016 at 12:19 PM Marvin Humphrey > wrote: > >> On Mon, Mar 21, 2016 at 9:57 AM, Craig Russell >> wrote: >>> There is a sorta technical reason for the Champion to be a member of the >> PMC >>> of the sponsor. >>> >>> I’d expect the Champion to subscribe to the private@ list and to have >>> binding votes on podling releases. These both require PMC membership. >>> >>> The alternative is to create two different “exceptions” that would allow >>> Champions to subscribe to private@ and to have binding release votes. >> >> These are legitimate concerns that would need to be dealt should such an >> unlikely scenario arise. However, I don't think we need to carve >> exceptions >> into policy here -- other creative solutions are available, like voting the >> Champion onto the Sponsor PMC. >> > > Moreso, the champion is responsible for bringing the project into the ASF. > The Champion holds no bearing on the project after that point. > > It sounds like what we're being pitched here is that the champion must stay > on to mentor the project. That hasn't been followed too much. I did not think this thread was discussing the role of the Champion, just the pre-requisites. http://incubator.apache.org/incubation/Roles_and_Responsibilities.html#Champion During incubation, the Champion: • Coordinates the creation and timely delivery of the podling's board reports. • Keeps an eye on the mentors' activity and takes action (ask for new mentors, talk to the Incubator PMC) if they don't seem to provide enough oversight or mentorship to the podling, The podling can elect a new Champion at any time, and must notify the Incubator PMC when that happens. The podling reports indicate who the current Champion is, and that information is kept up to date in the Incubator PMC's records (currently content/podlings.xml@champion). Craig > > >> >> And I'd like to take this opportunity to make a more general point: >> >> Policy should be simple. >> >> There are many reasons that policy should be simple, and I'm sure others >> will >> be happy to weigh in with their own favorites. But for me, this is the >> most compelling: >> >> Complexity harms newcomers. >> >> Right now, Apache's rules are so complex that we are all in perpetual >> violation. You can't even know what all the rules are! >> >> In such an environment, success is dependent, not on your own ability, but >> on >> securing alliances with powerful insiders who can help you bend or break >> the rules. >> >> This state of affairs is not worthy of our core principles. Particularly >> since the ASF does not exercise technical control over its projects, what >> we >> do here is not really that complicated. >> >> Apache, and the Incubator in particular, welcomes newcomers. It should be >> possible for a newcomers to discover and follow our rules largely through >> their own efforts. >> >> Of course a rejoinder to "Policy should be simple" is "As simple as >> possible >> and no simpler". But how close are we to "as simple as possible"? >> >> Marvin Humphrey >> >> - >> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org >> For additional commands, e-mail: general-h...@incubator.apache.org >> >> Craig L Russell Architect craig.russ...@oracle.com P.S. A good JDO? O, Gasp! - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Policy should be simple (was "Allowed Champions on podlings")
On Tue, Mar 22, 2016 at 12:19 PM Marvin Humphrey wrote: > On Mon, Mar 21, 2016 at 9:57 AM, Craig Russell > wrote: > > There is a sorta technical reason for the Champion to be a member of the > PMC > > of the sponsor. > > > > I’d expect the Champion to subscribe to the private@ list and to have > > binding votes on podling releases. These both require PMC membership. > > > > The alternative is to create two different “exceptions” that would allow > > Champions to subscribe to private@ and to have binding release votes. > > These are legitimate concerns that would need to be dealt should such an > unlikely scenario arise. However, I don't think we need to carve > exceptions > into policy here -- other creative solutions are available, like voting the > Champion onto the Sponsor PMC. > Moreso, the champion is responsible for bringing the project into the ASF. The Champion holds no bearing on the project after that point. It sounds like what we're being pitched here is that the champion must stay on to mentor the project. That hasn't been followed too much. > > And I'd like to take this opportunity to make a more general point: > > Policy should be simple. > > There are many reasons that policy should be simple, and I'm sure others > will > be happy to weigh in with their own favorites. But for me, this is the > most compelling: > > Complexity harms newcomers. > > Right now, Apache's rules are so complex that we are all in perpetual > violation. You can't even know what all the rules are! > > In such an environment, success is dependent, not on your own ability, but > on > securing alliances with powerful insiders who can help you bend or break > the rules. > > This state of affairs is not worthy of our core principles. Particularly > since the ASF does not exercise technical control over its projects, what > we > do here is not really that complicated. > > Apache, and the Incubator in particular, welcomes newcomers. It should be > possible for a newcomers to discover and follow our rules largely through > their own efforts. > > Of course a rejoinder to "Policy should be simple" is "As simple as > possible > and no simpler". But how close are we to "as simple as possible"? > > Marvin Humphrey > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > >
Re: Advice on binary NOTICE
On Sun, Mar 20, 2016 at 8:34 PM, Justin Mclean wrote: > It simple I like that. :) As expressed in another thread, I believe simple rules are really helpful for newcomers. > It will also tend to make NOTICE > files larger and that has a flow on effect to downstream projects. This is only one source of NOTICE content. We still need to emphasize leaving out other extraneous material. > Would’t it be better for incubating projects to just ask IPMC or legal for > help when putting together NOTICE files? There are other legal issues which people will still need to ask about; I don't that NOTICE is so interesting that the long and involved conversations we have about it today are especially beneficial. So making it more mechanical is desirable. Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Policy should be simple (was "Allowed Champions on podlings")
is this just personal catharsis? On Tue, Mar 22, 2016 at 4:19 PM, Marvin Humphrey wrote: > On Mon, Mar 21, 2016 at 9:57 AM, Craig Russell > wrote: >> There is a sorta technical reason for the Champion to be a member of the PMC >> of the sponsor. >> >> I’d expect the Champion to subscribe to the private@ list and to have >> binding votes on podling releases. These both require PMC membership. >> >> The alternative is to create two different “exceptions” that would allow >> Champions to subscribe to private@ and to have binding release votes. > > These are legitimate concerns that would need to be dealt should such an > unlikely scenario arise. However, I don't think we need to carve exceptions > into policy here -- other creative solutions are available, like voting the > Champion onto the Sponsor PMC. > > And I'd like to take this opportunity to make a more general point: > > Policy should be simple. > > There are many reasons that policy should be simple, and I'm sure others will > be happy to weigh in with their own favorites. But for me, this is the > most compelling: > > Complexity harms newcomers. > > Right now, Apache's rules are so complex that we are all in perpetual > violation. You can't even know what all the rules are! > > In such an environment, success is dependent, not on your own ability, but on > securing alliances with powerful insiders who can help you bend or break > the rules. > > This state of affairs is not worthy of our core principles. Particularly > since the ASF does not exercise technical control over its projects, what we > do here is not really that complicated. > > Apache, and the Incubator in particular, welcomes newcomers. It should be > possible for a newcomers to discover and follow our rules largely through > their own efforts. > > Of course a rejoinder to "Policy should be simple" is "As simple as possible > and no simpler". But how close are we to "as simple as possible"? > > Marvin Humphrey > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Policy should be simple (was "Allowed Champions on podlings")
On Mon, Mar 21, 2016 at 9:57 AM, Craig Russell wrote: > There is a sorta technical reason for the Champion to be a member of the PMC > of the sponsor. > > I’d expect the Champion to subscribe to the private@ list and to have > binding votes on podling releases. These both require PMC membership. > > The alternative is to create two different “exceptions” that would allow > Champions to subscribe to private@ and to have binding release votes. These are legitimate concerns that would need to be dealt should such an unlikely scenario arise. However, I don't think we need to carve exceptions into policy here -- other creative solutions are available, like voting the Champion onto the Sponsor PMC. And I'd like to take this opportunity to make a more general point: Policy should be simple. There are many reasons that policy should be simple, and I'm sure others will be happy to weigh in with their own favorites. But for me, this is the most compelling: Complexity harms newcomers. Right now, Apache's rules are so complex that we are all in perpetual violation. You can't even know what all the rules are! In such an environment, success is dependent, not on your own ability, but on securing alliances with powerful insiders who can help you bend or break the rules. This state of affairs is not worthy of our core principles. Particularly since the ASF does not exercise technical control over its projects, what we do here is not really that complicated. Apache, and the Incubator in particular, welcomes newcomers. It should be possible for a newcomers to discover and follow our rules largely through their own efforts. Of course a rejoinder to "Policy should be simple" is "As simple as possible and no simpler". But how close are we to "as simple as possible"? Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Wiki Access for Apache Incubator Proposal.
Thanks Marvin, That was blazingly fast, much appreciated! Best, Arthur On Mon, Mar 21, 2016 at 3:13 PM, Marvin Humphrey wrote: > On Mon, Mar 21, 2016 at 3:05 PM, Arthur Wiedmer > wrote: > > > Could I please be added to the Incubator Wiki Contributors Group? > > My wiki login is ArthurWiedmer. > > > > I am currently helping draft our proposal for Incubation located here: > > https://wiki.apache.org/incubator/AirflowProposal > > Done, enjoy! > > Marvin Humphrey > > - > To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org > For additional commands, e-mail: general-h...@incubator.apache.org > >
