Is there any chance you might consider making .jigdo files
for your ISO releases?
With the current stream of releases, I would imagine a huge
waste of bandwith downloading complete ISO's instead of
just the differences between releases.
Jigdo solves this issue by looking inside the ISO and only
downloading changes and generating the target ISO itself.
Saves loads of bandwith on frequent releases like Trixbox.
Thank you for your consideration.
That's kinda silly.. ISO's are for new installs. You should be using Yum to get updates, which is the whole point of it.
No it is not silly.
Many folks download each release and install it, this needs to happen to find the bugs.
Jigdo is a really tool for ISO dev teams to use or really for anyone with any big file; such a VMware / XEN / MPegs to distro the file(s) can be spread out over many servers at a time.
If we where using Jigdo the masses would be sucking down much less and the dev would be cleaner and easier to "fix on the fly".
The only issue is it is all cmd line and no GUI for it (or there was not one back in 05)
It makes sense to only download the changes in a base ISO install, one would still get
a ISO install but only download a small part of the iso most of the time, the core OS does not change much between most releases
In the older days the AAH / TB had the ISO build scripts in the iso and Jidgo made it so we could use a home cable modem to feed the dev team updates to the ISO without server up a 650 meg ISO
downloading a 70 meg tar file and few small scripts / rpm's is better than a 650 meg ISO any day.
Soon this will not really matter as we will be using a small boot iso and installing the bits and pieces from a download script.
While you are at it, an rsync server would be nice, and for you and us linux geeks would cut down substantially on the the bandwidth used.
+1
Well I just wanted suggest one of the probably scores of ways to
avoid fetching 500+ Mb of data just to get the newest release...
So your suggestions are as good as mine :)
Any feedback from the ISO packager?
Considering we have only had the one request from you, most likely because most people can suck down an ISO these days in about 20-30 minutes versus maybe being able to pull down this patch file, then run some other process to build the new ISO, there probably isn't any time savings there and I bet very few, if any, people are buying for bandwidth based on the amount downloaded. Not saying it isn't a valid suggestion, I just don't see anyone else asking for it.
Kerry here in Australia there are no Unlimited plans most ISP's go up to a maximum of 80GB then shaping to dialup speed occurs
My Intenet access costs my AU$70 per month for 20GB and thats for a 1500/256 ADSL connection 80GB plan is AU$160
Not that Trixbox adds to my quota my ISP has a mirror that Mirrors trixbox ISO's and dont count towards quota.
Kerry,
With rsync, you can take an old ISO image, rename it to the new image and then rsync it with your servers and presto, you have the new image. If you made 80k of changes in the ISO, then the download is only 80K.
I use rsync for offsite backup for my customers. I am backing up a customer with 10 Gig's of data over DSL Internet connection to their remote location in about 30 minutes. I have another with about 4 gigs of data and it take about 10 minutes. It only sends the changes.
Rsync is the cat's meow for updating thing like ISO's.
Sounds like an interesting idea, heck why download 650meg if you can just download 60-70 and be current. I never realized rsync could update an iso from one rev to another, I'll have to take a peek at that one.
If sorceforge doesn't support stuff like this, I am guessing (unless Fonality was against it) that it should be easy enough for someone, or a small group so setup such a mirror for people to use to get updates in that fashion..
I just checked the log for one customer. We backed up 3 gigs of data in 33 seconds, over the Internet using rsync. The actual data moved was 1 meg.
\\\off topic///
I am pulling this post as it is not necessarily productive.....
[qupte]It seems like somethings aren't done due to lack of knowledge or skill, The nice thing about Community is someone knows how and will help if you get over your self and allow it.[/qupte]
When have I not allowed anyone to help? We take tons of suggestions on every build and incorporate them and yes, I am not jumping on this particular issue because I do not have a good technical solution at hand.
There is allot of demand apparently for some sort of sync method weather jigdo, rsync. or whatever. People just know where asking for things get them as demonstrated in this thread
This thread is the first time its ever been brought up and less than half a dozen people are saying it would really be a benefit. That does not amount to "allot of demand". I don't understand the second sentance there.
its either no we don't need that or thats a freepbx issue you know what show that this is truly community software and let the community have it. or change the name to trixbox awoth (are way or the highway) with a side of nop(not our problem). This is how community projects work, you let the community build it and you give back to the community.
Again, I am not following your train of thought here. Should we be adding features and fixing bugs in all the other software projects that are included in trixbox? Where is the community building anything? It is an exceptionally rare time that anyone contributes anything to trixbox or freepbx and when someone offers a feature suggestion that really belongs in FreePBX, well...what are you suggesting we do about it? Nobody in the community steps up and creates it.
Let people fix the things you either dont have the skill or time to fix
We have a history of this already, most recently with faxing. The people who knew how to get it to work helped us to fix the issues and make faxing more reliable than ever.
and fix the things that you do for other projects you associate with. Biggest thing Your not flipping politicians quit making excuses and talking and circles and just do it
I don't understand what you are saying here. I don't follow the first sentance and I am not going to "just do it" for something that has limited value or would divert the very few resources I have off of finishing 2.4 which has a far greater appeal than solving a download problem that only a few people have requested. Is that an excuse, you bet it is. I have 1 engineer on my staff that has the skill to work on that and he will not be free for the forseeable future. If there is a better way of uploading to sourceforge, let me know. If someone has some more info on rsync, let me know, but to the best of MY knowledge it will sync files that are new or changed and would end up sending the entire ISO and not a diff of the ISO.
--
If someone has some more info on rsync, let me know, but to the best of MY knowledge it will sync files that are new or changed and would end up sending the entire ISO and not a diff of the ISO.
This was also my impression of rsync, and that if the ISO was a single file, it would sent the entire 650meg all over again. If this is not the case with rsync, that is a really cool feature of rsync that I have sure overlooked for a lot of years!
I would love to hear more about this, or any references to information on it, and how to configure it to do as mentioned. Heck I'd even be willing to consider setting it up for people to access the image if desired, and it worked as mentioned above by John..
Regards...
rsync can send only the delta within a file. I use it every night with sending changes of an 25 GB exchange database to an offsite location. This is the basis of products like veritas replication exec. I use that at 1 location, and a free product at another. Works great.
OK, I ran the test.
I downloaded trixbox-2.2.9.iso to my web server with wget. Then, I copied trixbox-2.2.4.iso to a directory on my file server here at my house. I then copied trixbox-2.2.4.iso into trixbox-2.2.9.iso, while perserving the date, to the same directory on my file server here. So now I have a seed file called trixbox-2.2.9.iso that is actually the trixbox-2.2.4.iso.
I wrote the following shell script to document the transfer:
#!/bin/bash
echo "The rsync job started at : `date`" > johnlog.txt
rsync -tv root@cohutta.com:/home/john/trixbox-2.2.9.iso . >> johnlog.txt
echo "The Rsync job finished at: `date`" >> johnlog.txt
And here is the johnlog.txt file:
The rsync job started at : Wed Dec 5 20:22:00 EST 2007
trixbox-2.2.9.iso
sent 167734 bytes received 257701674 bytes 191653.22 bytes/sec
total size is 571285504 speedup is 2.22
The Rsync job finished at: Wed Dec 5 20:44:25 EST 2007
The MD5's matched on both copies when I got done.
In a nutshell, I received the file in 22 minutes on a home DSL connection that is fairly busy right now. I downloaded about 257 megs of data, or about 1/2 of the size of the ISO. I suspect this would have been quicker if I had used a newer seed file. Obviously, a lot has changed from 2.2.4 to 2.2.9.
I hope this makes a difference in whether an rsync server could be made available. I will help with its management but I don't have the available bandwidth to support it at my webserver.
Rsync rocks folks!!!!
A few questions here:
1) Can this work off the sourceforge repository?
2) If not, how can I track the number of downloads?
3) If I published a link to the ISO, people could just download the ISO from that link instead of from sourceforge, this would be somewhat self-defeating from my point of view, anything I could do about that?
4) Any special requirements besides of the server?
My biggest concerns are monitoring the downloads, this is a key metric we look at to measure growth with, if I can't measure the downloads via this message I will never get management to sign off on a box. I would also need to know that more than a dozen people would ever use it.
I live in a country area of Queensland in Australia and the absolute best Broadband I can get is a is a wireless link at 512/512KB - please don't laugh I know it is a sad state of affairs.
Cheers
Garry
Kerry,
I don't know all of the answers.
I am pretty sure Sourceforge had an rsync server at one time.
I have never thought about tracking metrics. I just worry about bandwidth: good point.
You can run rsync from a linux server. It automatically installs on a lot of Linux versions, including trixbox.
Let me know if I can help accomplish this.
If someone can look into sourceforge support that would be a huge help.
Not sure if this is what your looking for kerry, but a quick google of sourceforge and rsync came up with this:
RsyncRsync access to the developer CVS content is provided, allowing projects and other users direct access to the raw CVS repository data, much like the deprecated nightly tarball service. The use of rsync means that there will no longer be a delay between what is in the repository and the data that a user can download for backups or major code refactoring. Rsync can be used to copy an entire project's repository (replace PROJECTNAME with the project UNIX name):
$ rsync -av rsync://PROJECTNAME.cvs.sourceforge.net/cvsroot/PROJECTNAME/* .
Note that you cannot rsync the entire cvsroot rsync module for performance reasons.
You can get to this page at: http://sourceforge.net/docs/E04/#rsync
Not sure if one can access the trixbox project this way or not...
Project name is still asteriskathome, would be interesting to see what can be done with that.
$ rsync -av rsync://asteriskathome.cvs.sourceforge.net/cvsroot/asteriskathome/* .
receiving file list ... done
CVSROOT/
CVSROOT/.#checkoutlist
CVSROOT/.#commitinfo
CVSROOT/.#config
CVSROOT/.#cvswrappers
CVSROOT/.#editinfo
CVSROOT/.#loginfo
CVSROOT/.#modules
CVSROOT/.#notify
CVSROOT/.#rcsinfo
CVSROOT/.#taginfo
CVSROOT/.#verifymsg
CVSROOT/checkoutlist
CVSROOT/checkoutlist,v
CVSROOT/commitinfo
CVSROOT/commitinfo,v
CVSROOT/config
CVSROOT/config,v
CVSROOT/cvswrappers
CVSROOT/cvswrappers,v
CVSROOT/editinfo
CVSROOT/editinfo,v
CVSROOT/history
CVSROOT/loginfo
CVSROOT/loginfo,v
CVSROOT/modules
CVSROOT/modules,v
CVSROOT/notify
CVSROOT/notify,v
CVSROOT/passwd
CVSROOT/rcsinfo
CVSROOT/rcsinfo,v
CVSROOT/readers
CVSROOT/taginfo
CVSROOT/taginfo,v
CVSROOT/val-tags
CVSROOT/verifymsg
CVSROOT/verifymsg,v
CVSROOT/writers
CVSROOT/Emptydir/
sent 940 bytes received 31643 bytes 5924.18 bytes/sec
total size is 29093 speedup is 0.89
Needless to say I didn't get an ISO, but it did access something..
Maybe you can put an ISO into the CVS so it can be gotten? Haven't never run a project on sourceforge, I honestly don't know, maybe others here do..
someone could write a script to track the login log using something like awk, and that could give you tracking, sourceforge may even track that already for you.
Was actually going to try and setup rsyncd on one of the servers in my co-lo. Not sure what I am doing wrong, but for some silly reason when I start rsyncd to serve up the ISO, it actually makes my entire server available from / on down. Guess I am going to have to do some more RTFM to figure out how to stop that.
If I get it going, I am happy to make it available to any to try and grab the ISO from, heck kerry can even have the logging info how many times it was downloaded if desired. It does seem like a neat idea if it makes for faster smaller updates on trix..
If someone can figure out all the details and how to report on it, we will look into putting up a server if there is enough demand.
Well I spent a little time playing with it here, and I guess I have it working..
As for logging Kerry, it looks like it writes out an entry like this if you enable transfer logging:
2007/12/11 03:41:01 [997] send noob.leadmon.net [38.103.9.9] trixbox () trixbox-2.3.0.8.76.iso 652800000
Not sure if that is what your looking for info wise.
As I put the ISO out there for testing, if anyone wants to try and grab it, or use it to get a copy, your more than welcome to, just rsync to trixbox.leadmon.net. I used the following command from another of my servers and it seemed to do fine.
rsync -v trixbox.leadmon.net::trixbox/trixbox-2.3.0.8.76.iso .
If this is all of any help that's super. When they release an updated image, I'll load it as well so I can test the different versions..
I also added the trixbox-2.3.2.68-2.3.0.7.iso image to the archive, so there was two to test against..
OK, I did try loading 2.3.0.7 to a server, and the followed up requesting 2.3.0.8, and without a doubt it did NOT transfer the whole file. Damn that is neat..
trixbox-2.3.2.68-2.3.0.7.iso
sent 119 bytes received 656970254 bytes 27956186.09 bytes/sec
total size is 656889856 speedup is 1.00
Then updating to .8:
trixbox-2.3.0.8.76.iso
sent 179565 bytes received 112565452 bytes 1039124.58 bytes/sec
total size is 652800000 speedup is 5.79
So without a doubt it was able to bit sync between the two images..
Very cool!
Nobody has tried it, but heck for anyone with a fast connection that cares to test it out, the machine is a Quad EMT64 server on a Gigabit connection to the net, so downloads should fly. Any logging I do get if anyone tries it, your most welcome to Kerry. I like the idea though, what a size/time reduction..
It all seems to work well for me from over here, but that was just testing locally, so if someone can do the same test remote, then we can see if it's really all a go or not. Heck if you release an update to 2.3/2.4 at some point later today as you said something about an announcement, I will make sure it's up to test/dl as well, so maybe someone will give it a try..
I just posted 2.2.10 and 2.3.0.9 should be later today.
OK, I just added 2.2.10 to the archive, and will add the 2.3.0.9 when it is made available..
So at this time the following is available via rSync at trixbox.leadmon.net:
trixbox-2.2.10.iso
trixbox-2.3.0.8.76.iso
trixbox-2.3.2.68-2.3.0.7.iso
It works like a charm. Have been using it to update ISO's for SMEServer for ages via sourceforge (www.smeserver.org)
In South Africa bandwidth is still very expensive so it is a better option to update an ISO rather than download a new one.
Member Since:
2007-10-07