Backing Up FTO Reports

[Tony Crocker]

The demise of the EpicSki Forum on first 3 days, then 2 weeks notice prompted a frenzy of activity to download and save content. viewtopic.php?f=10&t=12408#p77562

I have now tested the https://www.httrack.com/ software on FTO. It was successful and far less time consuming than for Epic. Part of this is that I already have a list of URL's for all TR's I have posted on FTO. It would/will be more time consuming to chase down other threads I might want to save, a similar process as I did with Epic before May 12.

The first screen of WinHTTRack prompts a project name and a directory where you want the files stored

In the left frame you can see my FTO Backup directory and the seven backup projects I ran

The next screen has a space for you to copy in the URL's you want to save in the current project, shown in blue here.

It looks like there's room for only a few URL's, but that box will take as many as you want.

Then click the "set options" button. Under the "Limits" tab I raoised the max transfer rate from the default 25,000 to 250,000.

At this point with Epic I hit "Next" then "Finish" and the program ran its course. FTO proved to be more problematic, as the program failed within 30 seconds and downloaded nothing. I grazed online to see if anyone else had a similar problem. Under "set options" you need to go to the "Browser ID" tab. FTO rejects the default, so you must choose the first one on the drop screen, referring to MSIE 6.0, shown in blue here:

WinHTTrack scrapes a huge amount of data, enough for your download to appear exactly as it does online. This means it will include frames, banners, ads, graphics, pictures, and linked URL's. The Epic downloads would run for 4=8 hours. The FTO downloads never stopped, so after 8 hours I interrupted, hit the tab to "finish current downloads" and the program would run for just a few minutes more.

[Admin]

And uses a metric crap ton of our bandwidth.

[Tony Crocker]

To no surprise there is a morass of directories and subdirectories within each project. But if you open the index file you get a list of your downloaded reports that you requested.

Click on one and you will see, stored on your C drive, the report exactly as it appears on FTO. Click on any picture, view image info, and you SHOULD see that the image has also been downloaded and stored to your C drive.

On the Epic download, HTTracks had some problems with threads in excess of 9 pages. Extra pages LOOKED LIKE they were downloaded but in reality were linked back to the online version. You COULD get those threads by downloading them just a handful at a time but it lenghtened the process quite a bit. Thankfully FTO has very few threads where length is excessive, and none on my TR's as far as I know.

On Epic, the URL's on the index contained the title name, so it was relatively easy to find something as long as you know which download project to check. FTO's URL's are opaque. In my case I have an enumerated list on a spreadsheet with the date and topic in other columns, so this is less of a hindrance to me than it might be to other people.

At this point I'm generally declaring victory. All of my FTO TR's including pictures are now on my C drive and not at the mercy of software platform changes, administrator error or indifference, etc. Patrick took this step nearly a decade ago, and the recent Epic debacle demonstrates that he was prescient in that regard.

Patrick told me at Mammoth that he has not reloaded all of his saved TR's to his blog. This can be a very tedious job, though obviously there is no deadline or time pressure to complete it. But given the morass of files that HTTrack scrapes, I was curious to run a test case of cleaning up one TR. I am pleased to report that the task is fairly efficient. Under the project name directory, there is a http://www.firstracksonline subdirectory. The core html file for each report is there. I open that file in Notepad++ and strip out the extraneous code used for frames, ads, banners, etc. Nearly all of this is separate from the text I'm trying to save.

The tedium of this process is usually involved in making the pictures display. If you download an FTO file manually, the picture links will all be back to the website and not to your C drive. This forces you to edit the html code manually for each picture. With HTTrack, the pictures are all in a download subdirectory of the http://www.firstracksonline subdirectory. So I created a download subdirectory under the directory I was using elsewhere in my C drive, then copied all the pictures from HTTracks' download subdirectory into it. Reloading my scrubbed version of the TR html file now displayed all the pics just fine. You don't even need to hunt down the specific pics for each TR. If they are in the proper "download " subdirectory, they will display.

[Tony Crocker]

And uses a metric crap ton of our bandwidth.

Compared to the thousands of hours of work that would be lost if FTO went the way of Epic, I have zero sympathy. The entire download process for all of my FTO TR's dating back to 2001 was completed between July 5 and this morning. At some point I will download enumerated threads of some other FTO topics, but that's likely to be fairly minor compared to the past three days.

And I am still reminded of upcoming platform changes to FTO every time I have to strip out this crap

" onclick="window.open(this.href);return false;" onclick="window.open(this.href);return false;" onclick="window.open(this.href);return false;

when editing a post.

[Admin]

Send your donations for bandwidth fees to:

PO Box 71171
Cottonwood Heights, UT 84171-1171

[Marc_C]

It's just possible that Tony and Patrick are the only ones who care about this.

[Tony Crocker]

It's just possible that Tony and Patrick are the only ones who care about this.

The archives are clearly the most important feature of FTO during the 7-8 months the News is silent because we have, what, maybe 10 active users left?

At any rate, Patrick is 100% correct that if you want your own content preserved, don't depend on someone else's server/platform for that.

[Tony Crocker]

Over the past 6 weeks I have expanded the reports uploaded to my own website: http://bestsnow.net/TRsFTO/triprpts.htm

Before I discovered the HTTrack program during the Epic shutdown, I had tediously uploaded just a few reports from Antarctica/Patagonia in 2011, Iceland 2015 and varied locations in the American Southwest. In 2017 I uploaded my Solar Eclipse TR's, including scanning pics and writing a TR for the only one that was not online already: http://bestsnow.net/TRsFTO/20021204ZimbabweEclipse.html

As far as ski reports were concerned, there are hundreds and I wanted some time to think about their organization and priority. So over the last six weeks I've rounded up ~50 by highlight category and uploaded them. For any TR saved by HTTrack this is a 10-20 minute job depending upon how pictures are labeled and organized. There is no recoding for pictures but HTTrack download directories are huge and it's best to narrow them down to the required pictures as much as possible before uploading.

During this process there were some highlight days that predated FTO and the digital camera era. A few had film pictures that could be scanned as with the 2002 eclipse report. These are probably of greatest interest:
Tuckerman Ravine, April 1990: http://bestsnow.net/TRsFTO/19900428Tuckerman.html
First remote lodge snowcat trip, Island Lake, February 1997: http://bestsnow.net/TRsFTO/19970223IslandLake.html
First remote lodge heliski trip, TLH, March 1998: http://bestsnow.net/TRsFTO/19980328TLH.html

If you wanted to addict a skier to cat/heli those first two trips in 1997 and 1998 were a good way to do it.

[jamesdeluxe]

Thanks for posting step-by-step instructions a couple years ago. That said -- while I understand going through this laborious HTTrack process if you're planning to repost reports to your own site, doing an old-fashioned copy/paste into a Word doc, followed by a couple minutes of easy reformatting works fine for me.

[Tony Crocker]

Copy/paste is one TR at a time. I use HTTrack at the end of the season by copying all the URL's into that web address box, let the computer run a few hours and get all of my TR's in one directory. At that point any TR is viewable on your own computer and will appear exactly as it did here on FTO. Setting up an HTTrack run takes just a few minutes of time once you figure out how to do it properly. I attempted to document that here in 2017.

So the only real work comes when you upload to another web site. This job seems manageable in bits and pieces based upon the 50+ reports I reloaded last fall.

My next step will be to have a linked list of selected TR's by region at the bottom of my regional list of areas skied, like this one for Europe: http://bestsnow.net/vft_euro.htm