Hi Wen,
Thank for the patch. I have some comments, (1) access global variant is higher cost, how about change below? cuGeom.log2CUSize == g_log2Size[m_param->minCUSize] (1 << cuGeom.log2CUSize) == m_param->minCUSize (2) looks mixed another fix on array index typo, suggest separate patch m_lumaIntraDir[absPartIdx] Regards, Chen At 2025-12-18 21:26:30, "Qiwei Wen" <[email protected]> wrote: >Dear Gurus, > >As you know, the HEVC spec allows NxN partition mode for intra-coded >CUs when the CU is of size MinCbSizeY. I assume either for coding >efficiency or historical reasons, x265 won't explore PART_NxN if the >minimum CU size is not 8. I'm working on a project that involves using >x265 to prototype intra-coding algorithms and it'd be ideal for my use >case if the limitation could be lifted. > >I realise that there are a few places in the code base that assume >that intra NxN implies a CU size of 8. Below is my attempt at fixing >it, the result of which is a non-local CABAC error, where the first >few intra NxN CU is coded correctly, but the CABAC state starts to >diverge to the point of making the bitstream undecodeable a few CUs >further down. > >I'd appreciate any pointers on if there are more places in the code >that makes the assumption and what the possible causes could be. > >Thanks, >Qiwei > >> diff --git a/source/encoder/analysis.cpp b/source/encoder/analysis.cpp >> index b219d5da4..4c42faba6 100644 >> --- a/source/encoder/analysis.cpp >> +++ b/source/encoder/analysis.cpp >> @@ -580,7 +580,7 @@ uint64_t Analysis::compressIntraCU(const CUData& >> parentCTU, const CUGeom& cuGeom >> checkIntra(md.pred[PRED_INTRA], cuGeom, SIZE_2Nx2N); >> checkBestMode(md.pred[PRED_INTRA], depth); >> >> - if (cuGeom.log2CUSize == 3 && m_slice->m_sps->quadtreeTULog2MinSize >> < 3) >> + if (cuGeom.log2CUSize == g_log2Size[m_param->minCUSize] && >> m_slice->m_sps->quadtreeTULog2MinSize < g_log2Size[m_param->minCUSize]) >> { >> md.pred[PRED_INTRA_NxN].cu.initSubCU(parentCTU, cuGeom, qp); >> checkIntra(md.pred[PRED_INTRA_NxN], cuGeom, SIZE_NxN); >> diff --git a/source/encoder/entropy.cpp b/source/encoder/entropy.cpp >> index d4bf73d53..9aecbb5e5 100644 >> --- a/source/encoder/entropy.cpp >> +++ b/source/encoder/entropy.cpp >> @@ -1328,7 +1328,7 @@ void Entropy::encodeTransform(const CUData& cu, >> uint32_t absPartIdx, uint32_t cu >> >> /* in each of these conditions, the subdiv flag is implied and not >> signaled, >> * so we have checks to make sure the implied value matches our >> intentions */ >> - if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N >> && log2CurSize == MIN_LOG2_CU_SIZE) >> + if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N >> && log2CurSize == g_log2Size[cu.m_encData->m_param->minCUSize]) >> { >> X265_CHECK(subdiv, "intra NxN requires TU depth below CU depth\n"); >> } >> @@ -1463,7 +1463,7 @@ void Entropy::encodeTransformLuma(const CUData& cu, >> uint32_t absPartIdx, uint32_ >> >> /* in each of these conditions, the subdiv flag is implied and not >> signaled, >> * so we have checks to make sure the implied value matches our >> intentions */ >> - if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N >> && log2CurSize == MIN_LOG2_CU_SIZE) >> + if (cu.isIntra(absPartIdx) && cu.m_partSize[absPartIdx] != SIZE_2Nx2N >> && log2CurSize == g_log2Size[cu.m_encData->m_param->minCUSize]) >> { >> X265_CHECK(subdiv, "intra NxN requires TU depth below CU depth\n"); >> } >> diff --git a/source/encoder/search.cpp b/source/encoder/search.cpp >> index 0522f52cc..3d61caabf 100644 >> --- a/source/encoder/search.cpp >> +++ b/source/encoder/search.cpp >> @@ -1541,8 +1541,8 @@ sse_t Search::estIntraPredQT(Mode &intraMode, const >> CUGeom& cuGeom, const uint32 >> { >> uint32_t bmode = 0; >> >> - if (intraMode.cu.m_lumaIntraDir[puIdx] != (uint8_t)ALL_IDX) >> - bmode = intraMode.cu.m_lumaIntraDir[puIdx]; >> + if (intraMode.cu.m_lumaIntraDir[absPartIdx] != (uint8_t)ALL_IDX) >> + bmode = intraMode.cu.m_lumaIntraDir[absPartIdx]; >> else >> { >> uint64_t candCostList[MAX_RD_INTRA_MODES]; > >-- >Qiwei Wen >Software Developer >Exablaze Pty Ltd >✆ 0430872689 >LinkedIn >_______________________________________________ >x265-devel mailing list >[email protected] >https://mailman.videolan.org/listinfo/x265-devel
_______________________________________________ x265-devel mailing list [email protected] https://mailman.videolan.org/listinfo/x265-devel
