Welcome
Welcome to refracta

You are currently viewing our boards as a guest, which gives you limited access to view most discussions and access our other features. By joining our free community, you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content, and access many other special features. In addition, registered members also see less advertisements. Registration is fast, simple, and absolutely free, so please, join our community today!

Squashfs Larger Than 4GB

Ask your questions here.

Squashfs Larger Than 4GB

Postby devils_debian » Sun Oct 21, 2012 8:19 pm

Hello
As the title suggests, I've hit a bit of a bump with Refracta Snaphot. :?
From the Refracta log: File iso live filesystem.squashfs is larger than 4GiB-1.
-allow-limited-size was not specified. There is no way do represent this file size. Aborting.


This problem occurred soon after my one and only snapshot success from my original Debian installation.
With only regular upgrades and a Exaile-Banshee swap out applied, the squashfs size has barely changed
from its approx 2.6GB (Gparted puts overall install disk usage at 14.73GiB out of 59.37GiB).

I decided to install my only snapshot to replace the original system. This went very well over all, with only minor operational niggles. Sadly though, the same problem occurred again...

In addition to running Bleachbit I decided to remove FPS 'Nexuiz' from the re-installed snapshot. This was a good 850MB (IIRC) reduction to the installed system. Again, same problem.
I then went crazy and removed all of 'X' and with it a huge swathe of applications and re-ran Refracta Snapshot from the CLI. Again, same issue reported in Refracta log. Very strange and not expected. :(

Snapshot appears to 'chug away' fine, until concluding the squashfs and creating the ISO.
I did a test 'save work=yes' attempt, which did leave me with the copied OS but nothing else except a 0 byte MD5 file..
Extra Tweaks
To avoid missing fstab errors I removed entries to leave it blank then commented out fstab in the excludes list. I also added to the excludes list: etc var alsa* to avoid 'alsactl store' errors on the live USB ISO. This worked nicely on different machines at work, detecting hardware well and booting to fully functional desktop.

This is so close to success, if I could nail the 'snapshot of a snapshot' issue. Any ideas?

Thanks
devils_debian
 
Posts: 8
Joined: Sat Oct 20, 2012 4:27 pm

Re: Squashfs Larger Than 4GB

Postby fsmithred » Mon Oct 22, 2012 1:17 am

Edit the script to add "-allow-limited-size" to the genisoimage options. You'll find it at either /usr/bin/refractasnapshot or /usr/bin/refractasnapshot-gui

Make this
Code: Select all
# create the iso file, make it isohybrid
# create md5sum file for the iso
echo "Creating CD image file..."

    genisoimage -r -J -l -D -o "$snapshot_dir"/"$filename" -cache-inodes \
    -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 \
    -boot-info-table iso/


look like this
Code: Select all
# create the iso file, make it isohybrid
# create md5sum file for the iso
echo "Creating CD image file..."

    genisoimage -r -J -l -D -o "$snapshot_dir"/"$filename" -cache-inodes \
    -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 \
    -boot-info-table -allow-limited-size iso/


That will be added, either as a default or as an option, in the next version of refractasnapshot.

The other thing you could do is add to the list of excluded items in /usr/lib/refractasnapshot/snapshot_exclude.list, which should be easier than uninstalling X.

And un-mount any additional filesystems that don't need to be copied. (like if you have a bunch of video on another hard drive that's mounted.)
User avatar
fsmithred
 
Posts: 2101
Joined: Wed Mar 09, 2011 9:13 pm

Re: Squashfs Larger Than 4GB

Postby devils_debian » Mon Oct 22, 2012 6:06 am

Thanks for the quick response. I will try this in the evening as soon as I return from work.
Regarding mounted filesystems. None of my other partitions were showing as mounted (greyed out devices on
desktop - Xfce). If they were, I would expect the that the mnt* media* entries to exclude them from the build anyway, no?

Out of interest, does my installed OS size seem reasonable at <15GiB (on 64GB SSD)?

Cheers.
devils_debian
 
Posts: 8
Joined: Sat Oct 20, 2012 4:27 pm

Re: Squashfs Larger Than 4GB

Postby fsmithred » Mon Oct 22, 2012 9:54 am

I'd say that 14GB is a huge installation. It's been awhile since I've had that much installed, and it's not so easy to do that in debian. Run Graphical Disk Map to see if you have some large files that can be deleted or excluded.

What is the "snapshot of a snapshot" problem? Not sure what you mean.

And yes, anything mounted under /mnt or /media is automatically excluded.
User avatar
fsmithred
 
Posts: 2101
Joined: Wed Mar 09, 2011 9:13 pm

Re: Squashfs Larger Than 4GB

Postby nadir » Mon Oct 22, 2012 12:48 pm

On the one hand:
14 Gigs for the system sounds very big to me.
I got 6.9 on this machine, on all others it is similar. That includes 1 Gig of backupdata at /root.
On the other hand:
When i used 64 i quickly reached 10 Gigs, which a similar setup. And i don't use an office-suite and such.
You might try apt-get clean. remove *gz files from /var/log. Remove unused kernels and headers. deborphan. If you did never run "apt-get clean" it will free a lot of space, and you can ignore the rest (it won't free much anyway. Take care with the rest. One typo and it means big trouble).

If i do a snapshot and want to make it CD size i try to make the installation not much bigger than 2 Gigs.
I got one for DVD, and i think it's size if 5 Gigs, the resulting iso being 1.5 Gig (or similar values, for CD i am sure, DVD i do seldom).
So i herd u liek mudkip?
User avatar
nadir
 
Posts: 1160
Joined: Wed Mar 09, 2011 4:18 am
Location: here

Re: Squashfs Larger Than 4GB

Postby devils_debian » Mon Oct 22, 2012 5:43 pm

Thanks for the script modification fsmithred. It did indeed work this time.
The ISO weighed in at 5.2GB ! Yes, that is huge, even for the stuff I have included.
It seems you were right nadir. I did have unwanted items on my system, found thanks
to Disk Analyser (Baobab). I had a Debian live build sitting around in /lib/live. Doh! Idiot (shakes head in shame).
The installed system now comes down to a svelte 6.2GB. Much more like it. :)
I'm going to snapshoot right after this post. Oh, by 'snapshot of a snapshot' I meant a snapshot of a newly
installed to disk, snapshot ISO. Hope that helps.
devils_debian
 
Posts: 8
Joined: Sat Oct 20, 2012 4:27 pm

Re: Squashfs Larger Than 4GB

Postby devils_debian » Mon Oct 22, 2012 10:08 pm

Ok guys, all sorted. Thank you very much for your help and kudos for creating the Refracta tools.
Typing this from my new snapshot (USB live), very pleased. Squashfs now back to 2.6GB as before. :D
Your scripts make easy work of hybrid stable (GDM), unstable (main), experimental (some) mixed bag installs.
I've really no doubt it will install well.
One tiny GUI presentation observation - Use of the window decoration title to show 'build stage' is truncated off due to small width of GUI size (on snapshot GUI).
Time to get 'evangelical' again and show what this 'baby' can do.

Cheers
Rob
devils_debian
 
Posts: 8
Joined: Sat Oct 20, 2012 4:27 pm

Re: Squashfs Larger Than 4GB

Postby fsmithred » Wed Oct 24, 2012 5:02 pm

Thanks for testing and reporting.

New version of refractasnapshot, gui and base, 9.0.6-1
http://distro.ibiblio.org/refracta/testing/
(didn't fix the window title yet, but thanks for that.)

Added '-allow-limited-size' to genisoimage.
Made '-comp xz' for mksquashfs an option in the config file.
User avatar
fsmithred
 
Posts: 2101
Joined: Wed Mar 09, 2011 9:13 pm

Re: Squashfs Larger Than 4GB

Postby stosteap » Sun Nov 08, 2015 10:51 pm

What about the current version of refractasnapshot?
It seems you are not using genisoimage anymore and I am hitting 4GB limit. Is there a way?
stosteap
 
Posts: 2
Joined: Sun Nov 08, 2015 10:49 pm

Re: Squashfs Larger Than 4GB

Postby fsmithred » Mon Nov 09, 2015 7:48 pm

We're using xorriso now in place of genisoimage. I think you're the first one to mention a problem with the size.

Try adding this option to the xorriso command.
Code: Select all
-file_size_limit off


If that doesn't work, try
Code: Select all
-compliance iso_9660_level=3


Or possibly, you need to have both options.


Around line 650 in the cli script or 770 in the gui script is this line:
Code: Select all
xorriso -as mkisofs -r -J -joliet-long -l ${isohybrid_opt} \
You can add the options at the end, before the final backslash, and leave a space before the backslash.

Please report your results. Thanks.
User avatar
fsmithred
 
Posts: 2101
Joined: Wed Mar 09, 2011 9:13 pm

Next

Return to Help

Who is online

Users browsing this forum: No registered users and 0 guests

cron
suspicion-preferred