I meant 4x4 partitions, obviously, sorry for the typo! On Fri, 19 Dec 2025 at 19:33, Richard <[email protected]> wrote: > > The problem is this > if (intraMode.cu.m_lumaIntraDir[absPartIdx] != (uint8_t)ALL_IDX) > bmode = intraMode.cu.m_lumaIntraDir[absPartIdx]; > Should not be changed like this. > This if branch is for analysis-load mode, in a normal "first time" prediction > this should always go to the else branch. > > > > Original > From: Qiwei Wen <[email protected]> > Date: 2025-12-19 05:53 > To: chen <[email protected]> > Cc: Development for x265 <[email protected]> > Subject: Re: [x265] Enabling intra NxN partition mode when MinCbSizeY > isgreater than 8 > > > Thanks Chen, will apply these suggestions as soon as I get past the cabac > issues. As I mentioned, the patch is missing some fixes and results in CABAC > errors whenever intra NxN is chosen in case that MinCbSizeY != 8. > > > Any suggestions on where else to look is greatly appreciated > > > Qiwei Wen > Software Developer > Exablaze Pty Ltd > ✆ 0430872689 > LinkedIn > > > > > > > > On Fri, Dec 19, 2025, 2:58 AM chen <[email protected]> wrote: > Hi Wen, > > > Thank for the patch. > > > I have some comments, > > > (1) access global variant is higher cost, how about change below? > cuGeom.log2CUSize == g_log2Size[m_param->minCUSize] > (1 << cuGeom.log2CUSize) == m_param->minCUSize > > > (2) looks mixed another fix on array index typo, suggest separate patch > m_lumaIntraDir[absPartIdx] > > > Regards, > Chen > > > > At 2025-12-18 21:26:30, "Qiwei Wen" <[email protected]> wrote: > >Dear Gurus, > > > >As you know, the HEVC spec allows NxN partition mode for intra-coded > >CUs when the CU is of size MinCbSizeY. I assume either for coding > >efficiency or historical reasons, x265 won't explore PART_NxN if the > >minimum CU size is not 8. I'm working on a project that involves using > >x265 to prototype intra-coding algorithms and it'd be ideal for my use > >case if the limitation could be lifted. > > > >I realise that there are a few places in the code base that assume > >that intra NxN implies a CU size of 8. Below is my attempt at fixing > >it, the result of which is a non-local CABAC error, where the first > >few intra NxN CU is coded correctly, but the CABAC state starts to > >diverge to the point of making the bitstream undecodeable a few CUs > >further down. > > > >I'd appreciate any pointers on if there are more places in the code > >that makes the assumption and what the possible causes could be. > > > >Thanks, > >Qiwei > > > >> diff --git a/source/encoder/analysis.cpp b/source/encoder/analysis.cpp > >> index b219d5da4..4c42faba6 100644 > >> --- a/source/encoder/analysis.cpp > >> +++ b/source/encoder/analysis.cpp > >> @@ -580,7 +580,7 @@ uint64_t Analysis::compressIntraCU(const CUData& > >> parentCTU, const CUGeom& cuGeom > >> checkIntra(md.pred[PRED_INTRA], cuGeom, SIZE_2Nx2N); > >> checkBestMode(md.pred[PRED_INTRA], depth); > >> > >> - if (cuGeom.log2CUSize == 3 && > >> m_slice->m_sps->quadtreeTULog2MinSize < 3) > >> + if (cuGeom.log2CUSize == g_log2Size[m_param->minCUSize] && > >> m_slice->m_sps->quadtreeTULog2MinSize < g_log2Size[m_param->minCUSize]) > >> { > >> md.pred[PRED_INTRA_NxN].cu.initSubCU(parentCTU, cuGeom, qp); > >> checkIntra(md.pred[PRED_INTRA_NxN], cuGeom, SIZE_NxN); > >> diff --git a/source/encoder/entropy.cpp b/source/encoder/entropy.cpp > >> index d4bf73d53..9aecbb5e5 100644 > >> --- a/source/encoder/entropy.cpp > >> +++ b/source/encoder/entropy.cpp > >> @@ -1328,7 +1328,7 @@ void Entropy::encodeTransform(const CUData& cu, > >> uint32_t absPartIdx, uint32_t cu > >> > >> /* in each of these conditions, the subdiv flag is implied and not > >> signaled, > >> * so we have checks to make sure the implied value matches our > >> intentions */ > >> - if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N > >> && log2CurSize == MIN_LOG2_CU_SIZE) > >> + if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N > >> && log2CurSize == g_log2Size[cu.m_encData->m_param->minCUSize]) > >> { > >> X265_CHECK(subdiv, "intra NxN requires TU depth below CU > >> depth\n"); > >> } > >> @@ -1463,7 +1463,7 @@ void Entropy::encodeTransformLuma(const CUData& cu, > >> uint32_t absPartIdx, uint32_ > >> > >> /* in each of these conditions, the subdiv flag is implied and not > >> signaled, > >> * so we have checks to make sure the implied value matches our > >> intentions */ > >> - if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N > >> && log2CurSize == MIN_LOG2_CU_SIZE) > >> + if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N > >> && log2CurSize == g_log2Size[cu.m_encData->m_param->minCUSize]) > >> { > >> X265_CHECK(subdiv, "intra NxN requires TU depth below CU > >> depth\n"); > >> } > >> diff --git a/source/encoder/search.cpp b/source/encoder/search.cpp > >> index 0522f52cc..3d61caabf 100644 > >> --- a/source/encoder/search.cpp > >> +++ b/source/encoder/search.cpp > >> @@ -1541,8 +1541,8 @@ sse_t Search::estIntraPredQT(Mode &intraMode, const > >> CUGeom& cuGeom, const uint32 > >> { > >> uint32_t bmode = 0; > >> > >> - if (intraMode.cu.m_lumaIntraDir[puIdx] != (uint8_t)ALL_IDX) > >> - bmode = intraMode.cu.m_lumaIntraDir[puIdx]; > >> + if (intraMode.cu.m_lumaIntraDir[absPartIdx] != (uint8_t)ALL_IDX) > >> + bmode = intraMode.cu.m_lumaIntraDir[absPartIdx]; > >> else > >> { > >> uint64_t candCostList[MAX_RD_INTRA_MODES]; > > > >-- > >Qiwei Wen > >Software Developer > >Exablaze Pty Ltd > >✆ 0430872689 > >LinkedIn > >_______________________________________________ > >x265-devel mailing list > >[email protected] > >https://mailman.videolan.org/listinfo/x265-devel > > > _______________________________________________ > x265-devel mailing list > [email protected] > https://mailman.videolan.org/listinfo/x265-devel
-- Qiwei Wen Software Developer Exablaze Pty Ltd ✆ 0430872689 LinkedIn _______________________________________________ x265-devel mailing list [email protected] https://mailman.videolan.org/listinfo/x265-devel
