[jira] [Commented] (TS-773) Traffic server has a hard limit of 512 gigabytes per RAW disk partition
[
https://issues.apache.org/jira/browse/TS-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13033705#comment-13033705
]
Leif Hedstrom commented on TS-773:
--
I've tested this with 2x 2TB disks, that I mounted using a USB adaptor on my
host (I'm assuming that shouldn't matter). In any case, trying it, I get a
segfault:
[May 15 09:25:58.284] Server {140021139494656} NOTE: clearing cache directory
'/dev/sdh 368640:244181087'
NOTE: Traffic Server received Sig 11: Segmentation fault
./bin/traffic_server - STACK TRACE:
Segmentation fault
(gdb) bt
#0 __memset_sse2 () at ../sysdeps/x86_64/memset.S:65
#1 0x006488f8 in vol_clear_init (d=0x7fffe40042c0) at Cache.cc:982
#2 0x0064b84e in Vol::clear_dir (this=value optimized out) at
Cache.cc:1014
#3 0x0064f928 in Vol::handle_header_read (this=0x7fffe40042c0,
event=value optimized out,
data=value optimized out) at Cache.cc:1487
#4 0x00652a15 in handleEvent (this=value optimized out, event=value
optimized out,
data=value optimized out) at ../../iocore/eventsystem/I_Continuation.h:146
#5 AIOCallbackInternal::io_complete (this=value optimized out, event=value
optimized out,
data=value optimized out) at ../../iocore/aio/P_AIO.h:80
#6 0x006dd7ee in handleEvent (this=0x7711a010, e=0xf9a070,
calling_code=1) at I_Continuation.h:146
#7 EThread::process_event (this=0x7711a010, e=0xf9a070, calling_code=1) at
UnixEThread.cc:140
#8 0x006de2d3 in EThread::execute (this=0x7711a010) at
UnixEThread.cc:232
#9 0x006dd42a in spawn_thread_internal (a=0xf83f90) at Thread.cc:85
#10 0x003c5de06ccb in start_thread (arg=0x7670f700) at
pthread_create.c:301
#11 0x003c5d2e0c2d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:115
Traffic server has a hard limit of 512 gigabytes per RAW disk partition
---
Key: TS-773
URL: https://issues.apache.org/jira/browse/TS-773
Project: Traffic Server
Issue Type: Bug
Components: Cache
Affects Versions: 2.1.8
Environment: Debian Lenny 5.0.8 2.6.34.7 x86_64
12 1.5TB harddrives for cache disks.
Reporter: David Robinson
Assignee: John Plevyak
Fix For: 2.1.9
Using 1.5TB harddrives as cache disks results in ATS only using 512GBs of the
disk. The disks are configured in RAW mode with no partition information.
storage.config is setup like this,
/dev/sda
/dev/sdb
/dev/sde
/dev/sdf
/dev/sdh
/dev/sdi
/dev/sdj
/dev/sdk
/dev/sdl
/dev/sdm
/dev/sdn
/dev/sdo
fdisk -l /dev/sdo
Disk /dev/sdo: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x
Partitioning a disk into 3 512G partition and adding then to storage.config
will make ATS use the entire 1.5TBs of space.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-773) Traffic server has a hard limit of 512 gigabytes per RAW disk partition
[ https://issues.apache.org/jira/browse/TS-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13033816#comment-13033816 ] John Plevyak commented on TS-773: - Well, I did predict failure :) In this case the problem was that the directory can now be more than 2GB in size, which exceeds an 'int'. The resulting patch touched lots of the system because it also means we can now read and write 2GB in a single go. I have submitted a patch and tested in a faked disk over 2TB (I had Store.cc lie about the size of the disk). Give it a go. Traffic server has a hard limit of 512 gigabytes per RAW disk partition --- Key: TS-773 URL: https://issues.apache.org/jira/browse/TS-773 Project: Traffic Server Issue Type: Bug Components: Cache Affects Versions: 2.1.8 Environment: Debian Lenny 5.0.8 2.6.34.7 x86_64 12 1.5TB harddrives for cache disks. Reporter: David Robinson Assignee: John Plevyak Fix For: 2.1.9 Using 1.5TB harddrives as cache disks results in ATS only using 512GBs of the disk. The disks are configured in RAW mode with no partition information. storage.config is setup like this, /dev/sda /dev/sdb /dev/sde /dev/sdf /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo fdisk -l /dev/sdo Disk /dev/sdo: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x Partitioning a disk into 3 512G partition and adding then to storage.config will make ATS use the entire 1.5TBs of space. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-773) Traffic server has a hard limit of 512 gigabytes per RAW disk partition
[
https://issues.apache.org/jira/browse/TS-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13033852#comment-13033852
]
Leif Hedstrom commented on TS-773:
--
Yep, that looks better, there are two warnings at startup, but I think that's
expected:
[May 15 18:45:44.543] Server {140432254875392} NOTE: reading directory
'/dev/sdi 368640:244181087'
[May 15 18:45:44.595] Server {140432254875392} NOTE: using directory A for
'/dev/sdh 368640:244181087'
[May 15 18:45:44.603] Server {140432254875392} NOTE: using directory A for
'/dev/sdi 368640:244181087'
[May 15 18:47:55.534] Server {140432248559360} WARNING: bad footer in cache
directory for '/dev/sdh 368640:244181087', clearing
[May 15 18:47:55.534] Server {140432248559360} NOTE: clearing cache directory
'/dev/sdh 368640:244181087'
[May 15 18:47:55.542] Server {140432247506688} WARNING: bad footer in cache
directory for '/dev/sdi 368640:244181087', clearing
[May 15 18:47:55.542] Server {140432247506688} NOTE: clearing cache directory
'/dev/sdi 368640:244181087'
It does take a while to initialize and startup with 4TB of USB storage ;).
After the startup has finished, things seems to work fine, and the cache size
report seems good:
proxy.process.cache.bytes_total=3995674296320
proxy.node.cache.bytes_total=3995674345472
proxy.node.cache.bytes_total_mb=3810572
proxy.node.cache.bytes_free=3995674345472
proxy.node.cache.bytes_free_mb=3810572
Thanks John!
Traffic server has a hard limit of 512 gigabytes per RAW disk partition
---
Key: TS-773
URL: https://issues.apache.org/jira/browse/TS-773
Project: Traffic Server
Issue Type: Bug
Components: Cache
Affects Versions: 2.1.8
Environment: Debian Lenny 5.0.8 2.6.34.7 x86_64
12 1.5TB harddrives for cache disks.
Reporter: David Robinson
Assignee: John Plevyak
Fix For: 2.1.9
Using 1.5TB harddrives as cache disks results in ATS only using 512GBs of the
disk. The disks are configured in RAW mode with no partition information.
storage.config is setup like this,
/dev/sda
/dev/sdb
/dev/sde
/dev/sdf
/dev/sdh
/dev/sdi
/dev/sdj
/dev/sdk
/dev/sdl
/dev/sdm
/dev/sdn
/dev/sdo
fdisk -l /dev/sdo
Disk /dev/sdo: 1500.3 GB, 1500301910016 bytes
255 heads, 63 sectors/track, 182401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x
Partitioning a disk into 3 512G partition and adding then to storage.config
will make ATS use the entire 1.5TBs of space.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-773) Traffic server has a hard limit of 512 gigabytes per RAW disk partition
[ https://issues.apache.org/jira/browse/TS-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13032061#comment-13032061 ] John Plevyak commented on TS-773: - I checked in what I hope is the fix. The cplist code needs to be cleaned up as it is not clear which counts are counting bytes/store blocks/disk volume blocks. WARNING: this fix required changing the disk structure which will result in a cache WIPE! Traffic server has a hard limit of 512 gigabytes per RAW disk partition --- Key: TS-773 URL: https://issues.apache.org/jira/browse/TS-773 Project: Traffic Server Issue Type: Bug Components: Cache Affects Versions: 2.1.8 Environment: Debian Lenny 5.0.8 2.6.34.7 x86_64 12 1.5TB harddrives for cache disks. Reporter: David Robinson Assignee: John Plevyak Fix For: 2.1.9 Using 1.5TB harddrives as cache disks results in ATS only using 512GBs of the disk. The disks are configured in RAW mode with no partition information. storage.config is setup like this, /dev/sda /dev/sdb /dev/sde /dev/sdf /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo fdisk -l /dev/sdo Disk /dev/sdo: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x Partitioning a disk into 3 512G partition and adding then to storage.config will make ATS use the entire 1.5TBs of space. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-773) Traffic server has a hard limit of 512 gigabytes per RAW disk partition
[ https://issues.apache.org/jira/browse/TS-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13032069#comment-13032069 ] Leif Hedstrom commented on TS-773: -- John: Do you foresee more fixes for this bug before v2.1.9, or does this complete this bug? Traffic server has a hard limit of 512 gigabytes per RAW disk partition --- Key: TS-773 URL: https://issues.apache.org/jira/browse/TS-773 Project: Traffic Server Issue Type: Bug Components: Cache Affects Versions: 2.1.8 Environment: Debian Lenny 5.0.8 2.6.34.7 x86_64 12 1.5TB harddrives for cache disks. Reporter: David Robinson Assignee: John Plevyak Fix For: 2.1.9 Using 1.5TB harddrives as cache disks results in ATS only using 512GBs of the disk. The disks are configured in RAW mode with no partition information. storage.config is setup like this, /dev/sda /dev/sdb /dev/sde /dev/sdf /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo fdisk -l /dev/sdo Disk /dev/sdo: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x Partitioning a disk into 3 512G partition and adding then to storage.config will make ATS use the entire 1.5TBs of space. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-773) Traffic server has a hard limit of 512 gigabytes per RAW disk partition
[ https://issues.apache.org/jira/browse/TS-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13032119#comment-13032119 ] John Plevyak commented on TS-773: - Unfortunately, the hosting code is pretty complicated and poorly written. I have tried to patch it, but it really should have a more comprehensive audit. If this patch fixes the immediate problem (which it seems to have, but I'd like independent confirmation) we can put off the audit till after 3.0. So I don't foresee more changes if we get the fix verified but I wouldn't be surprised if there was still a problem under some circumstances... until a full audit. Traffic server has a hard limit of 512 gigabytes per RAW disk partition --- Key: TS-773 URL: https://issues.apache.org/jira/browse/TS-773 Project: Traffic Server Issue Type: Bug Components: Cache Affects Versions: 2.1.8 Environment: Debian Lenny 5.0.8 2.6.34.7 x86_64 12 1.5TB harddrives for cache disks. Reporter: David Robinson Assignee: John Plevyak Fix For: 2.1.9 Using 1.5TB harddrives as cache disks results in ATS only using 512GBs of the disk. The disks are configured in RAW mode with no partition information. storage.config is setup like this, /dev/sda /dev/sdb /dev/sde /dev/sdf /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo fdisk -l /dev/sdo Disk /dev/sdo: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x Partitioning a disk into 3 512G partition and adding then to storage.config will make ATS use the entire 1.5TBs of space. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (TS-773) Traffic server has a hard limit of 512 gigabytes per RAW disk partition
[ https://issues.apache.org/jira/browse/TS-773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13031478#comment-13031478 ] David Robinson commented on TS-773: --- Debug log of all 12 disks http://pastebin.com/S343UjCV Traffic server has a hard limit of 512 gigabytes per RAW disk partition --- Key: TS-773 URL: https://issues.apache.org/jira/browse/TS-773 Project: Traffic Server Issue Type: Bug Components: Cache Affects Versions: 2.1.8 Environment: Debian Lenny 5.0.8 2.6.34.7 x86_64 12 1.5TB harddrives for cache disks. Reporter: David Robinson Using 1.5TB harddrives as cache disks results in ATS only using 512GBs of the disk. The disks are configured in RAW mode with no partition information. storage.config is setup like this, /dev/sda /dev/sdb /dev/sde /dev/sdf /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo fdisk -l /dev/sdo Disk /dev/sdo: 1500.3 GB, 1500301910016 bytes 255 heads, 63 sectors/track, 182401 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0x Partitioning a disk into 3 512G partition and adding then to storage.config will make ATS use the entire 1.5TBs of space. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
