[nuttx] branch master updated: fs/fat: Fix undefined behavior in signed integer overflow check

[xiaoxiang]

This is an automated email from the ASF dual-hosted git repository.

xiaoxiang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nuttx.git


The following commit(s) were added to refs/heads/master by this push:
     new 7d13f5eb78 fs/fat: Fix undefined behavior in signed integer overflow 
check
7d13f5eb78 is described below

commit 7d13f5eb78de2f13ceaa422325771e09cad02c73
Author: Mingjie Shen <shen...@purdue.edu>
AuthorDate: Mon Jun 26 00:36:28 2023 -0400

    fs/fat: Fix undefined behavior in signed integer overflow check
    
    Testing for overflow by adding a value to a variable to see if it "wraps
    around" works only for unsigned integer values, because signed overflow
    has undefined behavior according to the C and C++ standards.
    
    Signed-off-by: Mingjie Shen <shen...@purdue.edu>
---
 fs/fat/fs_fat32.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/fs/fat/fs_fat32.c b/fs/fat/fs_fat32.c
index 7f30d6d570..14238cd7aa 100644
--- a/fs/fat/fs_fat32.c
+++ b/fs/fat/fs_fat32.c
@@ -44,6 +44,16 @@
 #include "inode/inode.h"
 #include "fs_fat32.h"
 
+/****************************************************************************
+ * Pre-processor Definitions
+ ****************************************************************************/
+
+#if defined(CONFIG_FS_LARGEFILE)
+#  define OFF_MAX INT64_MAX
+#else
+#  define OFF_MAX INT32_MAX
+#endif
+
 /****************************************************************************
  * Private Function Prototypes
  ****************************************************************************/
@@ -764,7 +774,7 @@ static ssize_t fat_write(FAR struct file *filep, FAR const 
char *buffer,
 
   /* Check if the file size would exceed the range of off_t */
 
-  if (ff->ff_size + buflen < ff->ff_size)
+  if (buflen > OFF_MAX || ff->ff_size > OFF_MAX - (off_t)buflen)
     {
       ret = -EFBIG;
       goto errout_with_lock;