Chris, for all the time you helped so far I have to really appologize I've led you a stray ... so, reson the subvolumes were deleted is nothing to do with btrfs it self, I'm using "Rockstor" to ease managment tasks. This tool / environment / distribution treats a singular btrfs FS as a "pool" ( something in line of zfs :/ ) and when one removes a pool from the system it will actually go and delete subvolumes from FS before unmounting it and removing reference of it from it's DB (yes a bit shiet I know). so I'm not blaming anybody here for disapearing subvolumes, it's just me commig back to belive in man kind to get kicked in the gonads by mankind stupidity.
ALSO by importing the fs to their "solution" is just actually mounting and walking the tree of subvolumes to to create all the references in local DB (for rockstor of course, still nothing to do with btrfs functionality). To be able to ïmport" I've had to remove before mentioned snpshots becouse imports script was timing out. So for a single subvolume (called physically "share") I was left with no snapshots (removed by me to make import not time out) and then this subvolume was removed when I was trying to remove a fs (pool) from a running system. I've polled both disks (2 disk fs raid1) and I'm trying to rescue as much data as I can. The question is, why suddenly when I removed the snapshots and (someone else removed) the subvolume, there is such a great gap in generations of FS (over 200 generations missing) and the most recent generation that actually can me touched by btrfs restore is over a month old. How to over come that ? On 11 December 2016 at 19:00, Chris Murphy <li...@colorremedies.com> wrote: > On Sun, Dec 11, 2016 at 10:40 AM, Tomasz Kusmierz > <tom.kusmi...@gmail.com> wrote: >> Hi, >> >> So, I've found my self in a pickle after following this steps: >> 1. trying to migrate an array to different system, it became apparent >> that importing array there was not possible to import it because I've >> had a very large amount of snapshots (every 15 minutes during office >> hours amounting to few K) so I've had to remove snapshots for main >> data storage. > > True, there is no recursive incremental send. > >> 2. while playing with live array, it become apparent that some bright >> spark implemented a "delete all sub-volumes while removing array from >> system" ... needles to say that this behaviour is unexpected to say al >> least ... and I wanted to punch somebody in face. > > The technical part of this is vague. I'm guessing you used 'btrfs > device remove' butt it works no differently than lvremove - when a > device is removed from an array, it wipes the signature from that > device. You probably can restore that signature and use that device > again, depending on what the profile is for metadata and data, it may > be usable stand alone. > > Proposing assault is probably not the best way to ask for advice > though. Just a guess. > > > > >> >> Since then I was trying to rescue as much data as I can, luckily I >> managed to get a lot of data from snapshots for "other than share" >> volumes (because those were not deleted :/) but the most important >> volume "share" prove difficult. This subvolume comes out with a lot of >> errors on readout with "btrfs restore /dev/sda /mnt2/temp2/ -x -m -S >> -s -i -t". >> >> Also for some reason I can't use a lot of root blocks that I find with >> btrfs-find-root .. >> >> To put some detail here: >> btrfs-find-root -a /dev/sda >> Superblock thinks the generation is 184540 >> Superblock thinks the level is 1 >> Well block 919363862528(gen: 184540 level: 1) seems good, and it >> matches superblock >> Well block 919356325888(gen: 184539 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919343529984(gen: 184538 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920041308160(gen: 184537 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919941955584(gen: 184536 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919670538240(gen: 184535 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920045371392(gen: 184532 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920070209536(gen: 184531 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920117510144(gen: 184530 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 <<<---- here >> stuff is gone >> Well block 920139055104(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920139022336(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920138989568(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920138973184(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920137596928(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920137531392(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920137515008(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920135991296(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920135958528(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920135925760(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920135827456(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920135811072(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920133697536(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920133664768(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920133337088(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920133206016(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920132976640(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920132878336(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920132845568(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920128045056(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920127946752(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920127897600(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920127881216(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920127864832(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920127848448(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 920127799296(gen: 184511 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919815143424(gen: 184508 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919814569984(gen: 184508 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919811866624(gen: 184508 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919803232256(gen: 184496 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919712989184(gen: 184455 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919711891456(gen: 184455 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919706271744(gen: 184455 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919703568384(gen: 184455 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919699865600(gen: 184455 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 919647092736(gen: 184455 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417026879488(gen: 184334 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 <<--- I can >> only use that with btrfs restore and it's month old >> Well block 1417025896448(gen: 184333 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417029484544(gen: 184332 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417029320704(gen: 184332 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417029173248(gen: 184332 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417027616768(gen: 184332 level: 0) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417024831488(gen: 184331 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417021079552(gen: 184330 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417014001664(gen: 184329 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417011970048(gen: 184328 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> Well block 1417009758208(gen: 184327 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: >> ... rest of lines omitted to not trash the list to much. >> >> this line: >> Well block 920117510144(gen: 184530 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> is where stuff is missing, > > A root tree is only kept around for about 2 minutes, so it's going to > be freed up and subject to being overwritten if there's no snapshot > to pin it in place. > >> >> and this line: >> Well block 1417026879488(gen: 184334 level: 1) seems good, but >> generation/level doesn't match, want gen: 184540 level: 1 >> is the only line that physically work with >> btrfs restore /dev/sda /mnt2/temp2/ -x -m -S -s -i -t > > That particular > >> >> >> as you can see there is a vast "generation hole" between 184530 and >> 184334 ... more 200 generations ... and this results in only being >> able to access data that is month old :( and some of it is strangelly >> corrupted :( > > You're leaving out a lot of information. If you have thousands of > snapshots, and one of those snapshots has the data in it you want, > then you shouldn't need to use btrfs restore. Btrfs restore is a > scraping tool to try to find data, with no assurance of success at > all, from freed up metadata and data blocks. That data certainly could > be corrupt because all or parts of it may have been overwritten > already. > > If the data is in a stable snapshot that has not been deleted, then > you should be able to find it and extract it with a normal mount. If > the file system can't be mounted normally then that's what you need to > look at first - so presumably it's not mounting normally; but does it > mount with usebackuproot? Or ro,usebackuproot? Or > ro,usebackuproot,degraded? > > > -- > Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
