It’s not that simple. When you write into a table it is not put directly in the partition file. Instead it goes into the write ahead log and only later, during a checkpoint, gets written to the partition file(s). There’s really no reason to look directly inside the partition files.
> On 17 Jan 2022, at 10:58, Surinder Mehra <[email protected]> wrote: > > Hi Stephen, > I was trying to verify how data is stored in partitions. Though i verified by > running local scan query on each node, i was wondering if we can open > partition files like we can read kafka partitions. > > On Mon, Jan 17, 2022, 16:24 Stephen Darlington > <[email protected] <mailto:[email protected]>> > wrote: > What’s the use case for reading the data inside the partition files? > >> On 17 Jan 2022, at 10:16, Surinder Mehra <[email protected] >> <mailto:[email protected]>> wrote: >> >> I still havent found a way to read partition files. Not sure if we can >> >> 2nd: i created sample application to test it. I was able to see records >> colocated as per "companyid/dept" combination. >> >> >> >> On Fri, Jan 14, 2022, 22:54 Surinder Mehra <[email protected] >> <mailto:[email protected]>> wrote: >> Team, >> Please reply >> >> On Thu, Jan 13, 2022, 23:17 Surinder Mehra <[email protected] >> <mailto:[email protected]>> wrote: >> Hi, >> 1. Is it possible to read data inside partition files stored in >> /work/db/nodeid/... >> 2. If we have company and employee cache, employee has deptt nsme field, can >> i define an affinity key on {companyId, deptname} to partition data by >> company and then by deptname >> I want to colocate all company and its dept data on single node so my >> queries can locate data locally for a given company or deptt. >> >> I want a composite affinity key on companyId and depttname because deptname >> can be same across companies so data colocation is not achieved as required >> >
