Re: [lisp] Review of the LISP-NEXGON draft
On Tue, Aug 2, 2022 at 1:16 PM Dino Farinacci wrote: > Dear Professors Darrell and Yu, > > ... > > Detections from each area are aggregated in algorithmically (location) > addressable shards. > > > > A consolidation process is applied to merge multiple detections from > multiple points of view, varying time-stamps, and varying detection and > localization errors. The consolidation process emerges the current state - > enumeration of the condition of each grid tile aggregated by the shard. > Both condition enumeration, data-clustering, and consolidation processing > applied on network edge computers are aligned with BDD research. > > One question, what if two detections, roughly at the same time, report > different visualizations? Is there a policy, such that if one detected > nothing and another detected an object, that you err on choosing there was > an object present? > > In general such policies are called "non-maximal suppression", and yes, it is standard to include various heuristics and/or learned decision fusion techniques to resolve unique detections. While this is still somewhat an area of research to perfect such techniques, they have been already widely deployed for over a decade in e.g., vehicle and pedestrian detectors, both within views and across multiple views. Each vendor will likely implement a specific policy at first, these policies will generally rely on the network delivering as many samples as possible with minimal disruption to the same AI context, ie EID in this case. > ... > > Therefore we believe that LISP@IETF.org is the appropriate review venue > for this draft. Please do not hesitate to contact us for further > discussion of this important topic. > > That is great news. We will make sure we contact you if we need any > questions answered about the use-case. But Sharon is very fluent with the > use-case so he answers most of our questions. > > Cheers and thanks again, > Dino > > Thanks/Cheers t ___ lisp mailing list lisp@ietf.org https://www.ietf.org/mailman/listinfo/lisp
Re: [lisp] Review of the LISP-NEXGON draft
Thanks for the reply Trevor. Dino > On Aug 5, 2022, at 2:56 PM, Trevor Darrell wrote: > > > > On Tue, Aug 2, 2022 at 1:16 PM Dino Farinacci wrote: > Dear Professors Darrell and Yu, > > ... > > Detections from each area are aggregated in algorithmically (location) > > addressable shards. > > > > A consolidation process is applied to merge multiple detections from > > multiple points of view, varying time-stamps, and varying detection and > > localization errors. The consolidation process emerges the current state - > > enumeration of the condition of each grid tile aggregated by the shard. > > Both condition enumeration, data-clustering, and consolidation processing > > applied on network edge computers are aligned with BDD research. > > One question, what if two detections, roughly at the same time, report > different visualizations? Is there a policy, such that if one detected > nothing and another detected an object, that you err on choosing there was an > object present? > > > In general such policies are called "non-maximal suppression", and yes, it is > standard to include various heuristics and/or learned decision fusion > techniques to resolve unique detections. While this is still somewhat an > area of research to perfect such techniques, they have been already widely > deployed for over a decade in e.g., vehicle and pedestrian detectors, both > within views and across multiple views. Each vendor will likely implement a > specific policy at first, these policies will generally rely on the network > delivering as many samples as possible with minimal disruption to the same AI > context, ie EID in this case. > > ... > > Therefore we believe that LISP@IETF.org is the appropriate review venue for > > this draft. Please do not hesitate to contact us for further discussion of > > this important topic. > > That is great news. We will make sure we contact you if we need any questions > answered about the use-case. But Sharon is very fluent with the use-case so > he answers most of our questions. > > Cheers and thanks again, > Dino > > > Thanks/Cheers > t > ___ lisp mailing list lisp@ietf.org https://www.ietf.org/mailman/listinfo/lisp
Re: [lisp] Review of the LISP-NEXGON draft
Dear Professors Darrell and Yu, > Professors Darrell and Yu are leading researchers in AI, Computer Vision, and > Autonomous Driving, and have pioneered open-source frameworks and datasets > for autonomous driving research. Darrell has been in the field for over > three decades, founded the UC Berkeley BAIR and BDD centers, and is the > second most highly cited scholar in autonomous driving and the ninth-most in > computer vision according to Google Scholar. Yu is a leading researcher of > his generation in the area of perception for autonomous vehicles and was > recently hired as a tenure-track Assistant Professor at ETH after completing > a Postdoc at UC Berkeley, where he led the development of deep learning > models for autonomous driving and oversaw the collection of the BDD100K > dataset, which has been widely adopted in industry and academia. We are honored you have put effort into making LISP better for your use-case. We really appreciate the feedback. > The draft describes network aggregation of detections made by vehicles with > AI cameras driving at speeds of between 0 to 50 meters per second. Detections > are marked, enumerated, and localized by the vehicle, and are snapped to a > geospatial grid tile based on the vehicle position and geo-perspective > calculation. The enumeration and localization specified by the draft are > feasible with a reasonable onboard vehicle computer and are consistent with > current research results from our labs at UC Berkeley and ETH Zurich. That is good to know. > Detections from each area are aggregated in algorithmically (location) > addressable shards. > > A consolidation process is applied to merge multiple detections from multiple > points of view, varying time-stamps, and varying detection and localization > errors. The consolidation process emerges the current state - enumeration of > the condition of each grid tile aggregated by the shard. Both condition > enumeration, data-clustering, and consolidation processing applied on network > edge computers are aligned with BDD research. One question, what if two detections, roughly at the same time, report different visualizations? Is there a policy, such that if one detected nothing and another detected an object, that you err on choosing there was an object present? > The formal geospatial grid used for localization and consolidated aggregation > is the H3geo.org hierarchical hexagonal grid, as it provides for clear tile > adjacency of the grid in each resolution level. This is a useful quality in > calculating perspective, propagating impact of conditions, and resolving > shard border-line detections. We believe these design decisions are > reasonable. Yes, Sharon presented this to the WG many times. We were convinced. ;-) > We understood the detection aggregation network is based on IETF LISP RFCs to > provide: > > (1) seamless (to vehicles) edge compute expansion-contraction of per street > activity Yes, the mapping system can map from an EID that describes different resolutions. > (2) geo privacy, preventing unwarranted vehicle tracking by geolocation > services Yes, again, the mapping system provides this service. It is centralized for management and also distributed for scale. > (3) seamless context switching crossing shards while driving, without DNS > disruption Yes, we call that EID-mobility. > (4) service and subscription continuity when switching carriers/wlan while > driving Yes, the mapping system was extended from initial design to provide a PubSub capability and we are now using it for many of the use-case designs. > (5) mobile queuing, and metro ethernet edge route coalescing: M Mbps X Few > 100GE Right, EIDs can be aggregated when they are encoded as a power-of-2 address. > (6) replication of push notifications, network join: Vehicles X Situations X > Locations Yes, PubSub. > Therefore we believe that LISP@IETF.org is the appropriate review venue for > this draft. Please do not hesitate to contact us for further discussion of > this important topic. That is great news. We will make sure we contact you if we need any questions answered about the use-case. But Sharon is very fluent with the use-case so he answers most of our questions. Cheers and thanks again, Dino ___ lisp mailing list lisp@ietf.org https://www.ietf.org/mailman/listinfo/lisp
[lisp] Review of the LISP-NEXGON draft
Dear LISP@IETF.org, This is a review of the draft available at https://datatracker.ietf.org/doc/html/draft-ietf-lisp-nexagon by Prof. Trevor Darrell of UC Berkeley and Prof. Fisher Yu of ETH Zurich, founders of the Berkeley Deep Drive Consortium (BDD; https://bdd-data.berkeley.edu/) and the largest academic driving dataset, BDD100K (https://bdd100k.com). Professors Darrell and Yu are leading researchers in AI, Computer Vision, and Autonomous Driving, and have pioneered open-source frameworks and datasets for autonomous driving research. Darrell has been in the field for over three decades, founded the UC Berkeley BAIR and BDD centers, and is the second most highly cited scholar in autonomous driving and the ninth-most in computer vision according to Google Scholar. Yu is a leading researcher of his generation in the area of perception for autonomous vehicles and was recently hired as a tenure-track Assistant Professor at ETH after completing a Postdoc at UC Berkeley, where he led the development of deep learning models for autonomous driving and oversaw the collection of the BDD100K dataset, which has been widely adopted in industry and academia. The draft describes network aggregation of detections made by vehicles with AI cameras driving at speeds of between 0 to 50 meters per second. Detections are marked, enumerated, and localized by the vehicle, and are snapped to a geospatial grid tile based on the vehicle position and geo-perspective calculation. The enumeration and localization specified by the draft are feasible with a reasonable onboard vehicle computer and are consistent with current research results from our labs at UC Berkeley and ETH Zurich. Detections from each area are aggregated in algorithmically (location) addressable shards. A consolidation process is applied to merge multiple detections from multiple points of view, varying time-stamps, and varying detection and localization errors. The consolidation process emerges the current state - enumeration of the condition of each grid tile aggregated by the shard. Both condition enumeration, data-clustering, and consolidation processing applied on network edge computers are aligned with BDD research. The formal geospatial grid used for localization and consolidated aggregation is the H3geo.org hierarchical hexagonal grid, as it provides for clear tile adjacency of the grid in each resolution level. This is a useful quality in calculating perspective, propagating impact of conditions, and resolving shard border-line detections. We believe these design decisions are reasonable. We understood the detection aggregation network is based on IETF LISP RFCs to provide: (1) seamless (to vehicles) edge compute expansion-contraction of per street activity (2) geo privacy, preventing unwarranted vehicle tracking by geolocation services (3) seamless context switching crossing shards while driving, without DNS disruption (4) service and subscription continuity when switching carriers/wlan while driving (5) mobile queuing, and metro ethernet edge route coalescing: M Mbps X Few 100GE (6) replication of push notifications, network join: Vehicles X Situations X Locations Therefore we believe that LISP@IETF.org is the appropriate review venue for this draft. Please do not hesitate to contact us for further discussion of this important topic. Kind Regards, Profs. Darrell and Yu tre...@eecs.berkeley.edu i...@yf.io ___ lisp mailing list lisp@ietf.org https://www.ietf.org/mailman/listinfo/lisp
