立即注册 登录
气象家园 返回首页

MDAmike的个人空间 http://bbs.06climate.com/?63836 [收藏] [复制] [分享] [RSS]

日志

大型机上批量下载FNL数据

已有 143 次阅读2020-10-9 10:48 |个人分类:数据下载

FNL数据的下载地址:http://rda.ucar.edu/datasets/ds083.2/#access
登陆sign in (没有账号就注册)
   GRIB1 6 HOURLY FILES 1999.07.30 to 2007.12.06  :这是2007年12月6号之前的数据,为grib1格式的数据
   GRIB2 6 HOURLY FILES 2007.12.06 to current     :这是2007年12月06号至今的数据,为grib2格式的数据
   下载数据,点击Web File Listing
   若是批量下载数据,则点击Complete File List
   例如下载2017年11月的数据为例,点击 GRIB2 2017,再勾选上11月的数据( GRIB2 2017.11 ),之后点击Csh Download Script,出现一个网页,内容如下:

#!/bin/csh
################################################################# 
# Csh Script to retrieve 120 online Data files of 'ds083.2', 
# total 2.22G. This script uses 'wget' to download data. 
# Highlight this script by Select All, Copy and Paste it into a file;
# make the file executable and run it on command line. 
# You need pass in your password as a parameter to execute 
# this script; or you can set an environment variable RDAPSWD 
# if your Operating System supports it. 
# Contact grace@ucar.edu (Grace Peng) for further assistance. #################################################################  
set pswd = $1 
if(x$pswd == x && `env | grep RDAPSWD` != '') then 
   set pswd = $RDAPSWD 
endif 
if(x$pswd == x) then 
   echo
   echo Usage: $0 YourPassword 
   echo  
  exit 1 
endif 
set v = `wget -V |grep 'GNU Wget ' | cut -d ' ' -f 3` 
set a = `echo $v | cut -d '.' -f 1` 
set b = `echo $v | cut -d '.' -f 2` 
if(100 * $a + $b > 109) then 
    set opt = 'wget --no-check-certificate' 
else 
    set opt = 'wget' 
endif 
set opt1 = '-O Authentication.log --save-cookies auth.rda_ucar_edu --post-data' 
set opt2 = "email=xxxxxxxxxx@163.com&passwd=$pswd&action=login" 
$opt $opt1="$opt2" https://rda.ucar.edu/cgi-bin/login set opt1 = "-N --load-cookies auth.rda_ucar_edu" 
set opt2 = "$opt $opt1 http://rda.ucar.edu/data/ds083.2/" 
set filelist = ( \ 
     grib2/2017/2017.11/fnl_20171101_00_00.grib2 \ grib2/2017/2017.11/fnl_20171101_06_00.grib2 \ 
     grib2/2017/2017.11/fnl_20171101_12_00.grib2 \ grib2/2017/2017.11/fnl_20171101_18_00.grib2 \
    grib2/2017/2017.11/fnl_20171102_00_00.grib2 \ grib2/2017/2017.11/fnl_20171102_06_00.grib2 \
    grib2/2017/2017.11/fnl_20171102_12_00.grib2 \ grib2/2017/2017.11/fnl_20171102_18_00.grib2 \
    grib2/2017/2017.11/fnl_20171103_00_00.grib2 \ grib2/2017/2017.11/fnl_20171103_06_00.grib2 \ 
    grib2/2017/2017.11/fnl_20171103_12_00.grib2 \ grib2/2017/2017.11/fnl_20171103_18_00.grib2 \ 
    grib2/2017/2017.11/fnl_20171104_00_00.grib2 \ grib2/2017/2017.11/fnl_20171104_06_00.grib2 \ 
    grib2/2017/2017.11/fnl_20171104_12_00.grib2 \ grib2/2017/2017.11/fnl_20171104_18_00.grib2 \ 
    grib2/2017/2017.11/fnl_20171105_00_00.grib2 \ grib2/2017/2017.11/fnl_20171105_06_00.grib2 \ 
    grib2/2017/2017.11/fnl_20171105_12_00.grib2 \ grib2/2017/2017.11/fnl_20171105_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171106_00_00.grib2 \ grib2/2017/2017.11/fnl_20171106_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171106_12_00.grib2 \ grib2/2017/2017.11/fnl_20171106_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171107_00_00.grib2 \ grib2/2017/2017.11/fnl_20171107_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171107_12_00.grib2 \ grib2/2017/2017.11/fnl_20171107_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171108_00_00.grib2 \ grib2/2017/2017.11/fnl_20171108_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171108_12_00.grib2 \ grib2/2017/2017.11/fnl_20171108_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171109_00_00.grib2 \ grib2/2017/2017.11/fnl_20171109_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171109_12_00.grib2 \ grib2/2017/2017.11/fnl_20171109_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171110_00_00.grib2 \ grib2/2017/2017.11/fnl_20171110_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171110_12_00.grib2 \ grib2/2017/2017.11/fnl_20171110_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171111_00_00.grib2 \ grib2/2017/2017.11/fnl_20171111_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171111_12_00.grib2 \ grib2/2017/2017.11/fnl_20171111_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171112_00_00.grib2 \ grib2/2017/2017.11/fnl_20171112_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171112_12_00.grib2 \ grib2/2017/2017.11/fnl_20171112_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171113_00_00.grib2 \ grib2/2017/2017.11/fnl_20171113_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171113_12_00.grib2 \ grib2/2017/2017.11/fnl_20171113_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171114_00_00.grib2 \ grib2/2017/2017.11/fnl_20171114_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171114_12_00.grib2 \ grib2/2017/2017.11/fnl_20171114_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171115_00_00.grib2 \ grib2/2017/2017.11/fnl_20171115_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171115_12_00.grib2 \ grib2/2017/2017.11/fnl_20171115_18_00.grib2 \       
   grib2/2017/2017.11/fnl_20171116_00_00.grib2 \ grib2/2017/2017.11/fnl_20171116_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171116_12_00.grib2 \ grib2/2017/2017.11/fnl_20171116_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171117_00_00.grib2 \ grib2/2017/2017.11/fnl_20171117_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171117_12_00.grib2 \ grib2/2017/2017.11/fnl_20171117_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171118_00_00.grib2 \ grib2/2017/2017.11/fnl_20171118_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171118_12_00.grib2 \ grib2/2017/2017.11/fnl_20171118_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171119_00_00.grib2 \ grib2/2017/2017.11/fnl_20171119_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171119_12_00.grib2 \ grib2/2017/2017.11/fnl_20171119_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171120_00_00.grib2 \ grib2/2017/2017.11/fnl_20171120_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171120_12_00.grib2 \ grib2/2017/2017.11/fnl_20171120_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171121_00_00.grib2 \ grib2/2017/2017.11/fnl_20171121_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171121_12_00.grib2 \ grib2/2017/2017.11/fnl_20171121_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171122_00_00.grib2 \ grib2/2017/2017.11/fnl_20171122_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171122_12_00.grib2 \ grib2/2017/2017.11/fnl_20171122_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171123_00_00.grib2 \ grib2/2017/2017.11/fnl_20171123_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171123_12_00.grib2 \ grib2/2017/2017.11/fnl_20171123_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171124_00_00.grib2 \ grib2/2017/2017.11/fnl_20171124_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171124_12_00.grib2 \ grib2/2017/2017.11/fnl_20171124_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171125_00_00.grib2 \ grib2/2017/2017.11/fnl_20171125_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171125_12_00.grib2 \ grib2/2017/2017.11/fnl_20171125_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171126_00_00.grib2 \ grib2/2017/2017.11/fnl_20171126_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171126_12_00.grib2 \ grib2/2017/2017.11/fnl_20171126_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171127_00_00.grib2 \ grib2/2017/2017.11/fnl_20171127_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171127_12_00.grib2 \ grib2/2017/2017.11/fnl_20171127_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171128_00_00.grib2 \ grib2/2017/2017.11/fnl_20171128_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171128_12_00.grib2 \ grib2/2017/2017.11/fnl_20171128_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171129_00_00.grib2 \ grib2/2017/2017.11/fnl_20171129_06_00.grib2 \  
   grib2/2017/2017.11/fnl_20171129_12_00.grib2 \ grib2/2017/2017.11/fnl_20171129_18_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171130_00_00.grib2 \ grib2/2017/2017.11/fnl_20171130_06_00.grib2 \ 
   grib2/2017/2017.11/fnl_20171130_12_00.grib2 \ grib2/2017/2017.11/fnl_20171130_18_00.grib2 \ 
while($#filelist > 0) 
  set syscmd = "$opt2$filelist[1]" 
  echo "$syscmd ..." 
  $syscmd 
  shift filelist 
end 

rm -f auth.rda_ucar_edu Authentication.log 
exit 0
-----------------------------------------------------------------------------------------------------------------------------------
其中,xxxxxxxxxx@163.com为登陆网站的用户邮箱
之后在大型机上新建一个文件,
   $vi download
击“i”,进入插入模式,将script内容复制粘贴到download中,在按“Esc”键,进入命令模式,输入“:wq!”,保存退出;
   $chmod 755 download  
在执行download之前,可以设置环境变量,避免每次输入密码;
   $vi ~/.bashrc
加入下面一行:
   export RDAPSWD=xxxxxx 
其中,xxxxxx为你登陆NCEP FNL数据网站的密码
改好后,“Esc”键,输入“:wq!”
   $source ~/.bashrc  使环境变量设置生效 
然后再执行download
   $./download 
就直接开始下载啦!

评论 (0 个评论)

facelist doodle 涂鸦板

您需要登录后才可以评论 登录 | 立即注册

Copyright ©2011-2014 bbs.06climate.com All Rights Reserved.  Powered by Discuz! (京ICP-10201084)

本站信息均由会员发表,不代表气象家园立场,禁止在本站发表与国家法律相抵触言论

返回顶部