Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Next revision Both sides next revision
server-administration:wget-recursive-ftp [2010-10-28 21:55:15]
garrett.plasky created
server-administration:wget-recursive-ftp [2010-10-28 21:57:45]
garrett.plasky Approved
Line 2: Line 2:
  
 We have found that using wget to recursively grab FTP contents is useful in the following situation: We have found that using wget to recursively grab FTP contents is useful in the following situation:
-  * You cannot use rsync or scp due to restricted or no shell access on the remote server+  * You cannot use <c>rsync</c> or <c>scp</c> due to restricted or no shell access on the remote server
   * You need to recursively get directories and globbing with <c>mget *</c> isn't working as expected   * You need to recursively get directories and globbing with <c>mget *</c> isn't working as expected
   * Transferring files to an intermediate workstation first is not feasible due to time/filesize constraints   * Transferring files to an intermediate workstation first is not feasible due to time/filesize constraints
  
-If all of these apply to you, then a recursive wget via FTP is probably the method for you. On the surface recursive wget is quite simple:+If all of these apply to you, then a recursive wget via FTP is probably your best bet. On the surface recursive wget is quite simple:
  
 <code console># wget -r ftp://user:pass@host.com/path/to/web/content/</code> <code console># wget -r ftp://user:pass@host.com/path/to/web/content/</code>
Line 14: Line 14:
 <code console># wget -r -nH --cut-dirs=4 ftp://user:pass@host.com/path/to/web/content/</code> <code console># wget -r -nH --cut-dirs=4 ftp://user:pass@host.com/path/to/web/content/</code>
  
-In the above example, <c>--cut-dirs</c> was set to 4 because we are transversing down 4 directories to get the source data, so be sure to modify that number based on your particular case. The <c>-nH</c> option specifies to not include the portion of the wget URL in the path as well.+In the above example, <c>--cut-dirs</c> was set to 4 because we are transversing down 4 directories to get the source data(<c>/path/to/web/content/</c>), so be sure to modify that number based on your particular case. The <c>-nH</c> option specifies to not include the host portion of the wget URL in the path as well.
  
 Finally, you can also add <c>-nc</c> to ensure that any locally existing files are not overwritten by duplicates coming in from the remote server. Finally, you can also add <c>-nc</c> to ensure that any locally existing files are not overwritten by duplicates coming in from the remote server.