Actually there is no API for that right now, Currently the tricky workaround is that you can use arrow[1] to read them as table and and append vertices to the chunk table.
The VertexBuilder is designed to build a graph from scratch, not supporting append vertices to current graph right now. [1] https://arrow.apache.org/docs/cpp/api/formats.html Best, Weibin Zeng On 2024/08/28 08:26:07 Muniraju Pulipalyam wrote: > > OK. What's the API to read the last chunk? Is it with VerticesCollection? > VerticesCollection seems to be very slow. I looked at mid_level _reader > sample, but that seems to be about one property. > > > ________________________________ > From: Weibin Zeng <[email protected]> > Sent: Wednesday, August 28, 2024 12:30:30 PM > To: [email protected] <[email protected]> > Subject: Re: Add a new vertex to an existing graph of vertices and edges. > > Hi，Muni, Currently graphar doesn't support appending vertices to a existed > chunk (which is a existed parquet or orc file) but new a chunk. So the best > way is read the chunk you want to append and append to the chunk in memory, > then write it out. > > I think this is the case that we need to address in the library, which no > need user to take care the process. I will open an issue for this feature. > > Thanks, > Weibin Zeng > > On 2024/08/28 03:05:54 Muniraju Pulipalyam wrote: > > Hello all/ Weibin, > > > > Can someone please help me with this query below? Say, I have a chunk size > > of 1024. If I have 5000 vertices in my graph, 4 chunks will be of 1024 > > vertices length and 5th chunk will be 904 vertices. When I try to write > > 5001st item, if I pass 5001 as the start vertex index to VerticeBuilder, it > > is overriding 904 elements with 1 element I am adding. Should I read the > > vertices of last chunk, add to the builder and then append back? > > > > Thanks, > > Muni > > > > ________________________________ > > From: Muniraju Pulipalyam > > Sent: Monday, August 26, 2024 8:49 PM > > To: [email protected] <[email protected]> > > Subject: Re: Add a new vertex to an existing graph of vertices and edges. > > > > Thanks Weibin. This seems to work only if vertex chunk size is 1, if I want > > to add 1 vertex to an existing graph with vertices when I have strong > > validation. When chunk size is 1 it creates new file and everything works. > > But if chunk size is not 1 , if I enable strong validation, add vertex > > fails with "the start vertex index <my index number> is not aligned with > > chunk size" in verticesbuulder.ccline 44. If I don't enable strong > > validation, then last chunk file is overwritten with 1 row data, when I > > call dump on vertexBuilder. I tried with 10 row vertices ,chunk size 5. > > 11th row succeeds with new chunk created. But, if I change chunk size to 4, > > then I get above validation error > > > > Sent from Outlook for > > Android<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2FAAb9ysg&data=05%7C02%7C%7C22d457b4dd024c368a5808dcc72f1c83%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638604252363995851%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=L9yQYw0Q6boAT5h6yIMnRVPBjLlDcdDIjIoOwj2nDf8%3D&reserved=0<https://aka.ms/AAb9ysg>> > > ________________________________ > > From: Weibin Zeng <[email protected]> > > Sent: Monday, August 26, 2024 8:46:40 AM > > To: [email protected] <[email protected]> > > Subject: Re: Add a new vertex to an existing graph of vertices and edges. > > > > You need to specify the start vertex index when you construct the > > `VerticesBuilder`[1] if you want append vertices to existed graph. In your > > senario, the start vertex index should be 5001. > > > > [1] > > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgraphar.apache.org%2Fdocs%2Fcpp%2Fclassgraphar_1_1builder_1_1_vertices_builder.html%23a8215ee434947314027aae3a7e10dac5b&data=05%7C02%7C%7C22d457b4dd024c368a5808dcc72f1c83%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638604252364002978%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=6zM%2Bp7Ff%2Bhq08Byrk%2BPYQMDIF2lL1VqIY0ENvUHRrlM%3D&reserved=0<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgraphar.apache.org%2Fdocs%2Fcpp%2Fclassgraphar_1_1builder_1_1_vertices_builder.html%23a8215ee434947314027aae3a7e10dac5b&data=05%7C02%7C%7C22d457b4dd024c368a5808dcc72f1c83%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638604252364005896%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=4VUgHmbYXqI3jGE8xLNEQKbmYc5Zm40s0z%2BNm8nhii4%3D&reserved=0><https://graphar.apache.org/docs/cpp/classgraphar_1_1builder_1_1_vertices_builder.html#a8215ee434947314027aae3a7e10dac5b> > > > > Best, > > Weibin Zeng > > > > On 2024/08/25 06:51:45 Muniraju Pulipalyam wrote: > > > I have an existing graph with 5000 vertices and 80000 edges. The vertices > > > are in 4 chunks. When I try to use verticesBuilder and create 5001st > > > vertex to add to existing collection, it is overriding existing vertex > > > chunks. I also tried WriteTable, but can't find a way to append. I saw > > > samples on mid-level and high-level writers and other samples where new > > > files are created in /tmp folder. I want to append a vertex to an > > > existing vertex collection. What API can I use to append a vertex to > > > vertex collection and write back to file? > > > > > > Thanks. > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
