• 设为首页
  • 点击收藏
  • 手机版
    手机扫一扫访问
    迪恩网络手机版
  • 关注官方公众号
    微信扫一扫关注
    迪恩网络公众号

cgaustin/mastodon

原作者: [db:作者] 来自: 网络 收藏 邀请

开源软件名称(OpenSource Name):

cgaustin/mastodon

开源软件地址(OpenSource Url):

https://github.com/cgaustin/mastodon

开源编程语言(OpenSource Language):

Shell 73.9%

开源软件介绍(OpenSource Introduction):

Mastodon

Tools for facilitating LCMAP data curation.

Features

  • Provide a CLI for directing data ingest to the IWDS
  • Handle parallelization of data ingest requests

Running

The LCMAP Mastodon application is deployed as a Docker container. All interactions are handled over HTTP.

docker run \
-v /workspace/data:/data \
-e "ARD_PATH=${ARD_PATH}" \
-e "ARD_HOST=${ARD_HOST}"\
-e "CHIPMUNK_HOST=${CHIPMUNK_HOST}" \
-e "DATA_TYPE=${DATA_TYPE}" \
-e "PARTITION_LEVEL=${PARTITION_LEVEL}" \
usgseros/lcmap-mastodon

Configuration

You need to mount a volume to your container at /data. This should be the base dir to where the ARD tarballs can be found

-v /localardpath/data:/data

And the following environment variables:

${ARD_PATH} - is used by a glob function to determine what ARD tarballs are available for a given Tile ID. The value is determined by the directory structure where the ARD is kept

Analysis Ready Data (ARD) are expected to be organized by Landsat Mission. From the mounted dir, the directory structure should mirror this: <mission>/ARD_Tile/<year acquired>/<region>/<HHH>/<VVV>/

HHH and VVV constituting the 3 digit tile-id. The H and V values DO NOT need to be included in your ${ARD_PATH} definition.

export ARD_PATH=/data/\{tm,etm,oli_tirs\}/ARD_Tile/*/CU/

${ARD_HOST} - is the host or IP address for the deployed lcmap-mastodon instance

${CHIPMUNK_HOST} - is the url to the deployed lcmap-chipmunk instance

${DATA_TYPE} - tells the lcmap-mastodon instance what kind of data it is working with. Valid values are "ard" and "aux".

${PARTITION_LEVEL} - determines the level of parallelization applied to the ingest process. For instance, setting this to 10 results in 10 simultaneous data ingest requests.

${AUX_HOST} - needs to be defined if ${DATA_TYPE} is defined as "aux". It is the host or ip address where auxiliary data is provided.

Optionally, you can define the following:

${INVENTORY_TIMEOUT} - defines, in milliseconds, the HTTP request timeout for inventory queries against lcmap-chipmunk Defaults to 120000 (2 minutes).

${INGEST_TIMEOUT} - defines, in milliseconds, the HTTP request timeout for ingest requests against lcmap-chipmunk. Defaults to 120000 (2 minutes).

CLI Interaction

Build an uberjar with leiningen, and export environment variables for the ${CHIPMUNK_HOST}, ${PARTITION_LEVEL}, ${DATA_TYPE} and ${ARD_HOST}. The ${ARD_HOST} is an instance of Mastodon running in server mode. It is this Mastodon instance which exposes the ARD over HTTP for ingest.

If you want to ingest Auxiliary data instead of ARD, you'll need to set the DATA_TYPE accordingly, and also define ${AUX_HOST}.

Optionally, you can export FROM_DATE and TO_DATE environment variables, to filter by year the ARD you're concerned with. Format is: YYYY.

To build the standalone jar file:

lein uberjar

With your jar built, and your environment setup

java -jar target/lcmap-mastodon-0.1.13-standalone.jar <6 digit tile id>

And follow the prompts. If you want to automatically ingest any previously uningested data, add -y after the tile id.

Development Clojure

To get a repl:

lein repl

To run the main function:

lein run

Testing

make runtests

Docker

Before building a new docker image, you'll need to create a new uberjar and transpile the clojurescript

lein uberjar
docker build -t < your tag here > .

Contributing

Before committing to this repository, run the following command.

git config --local commit.template .gitmessage

This will add the LCMAP commit template to git commit.

jira/lcmap-xxxx: Title
Description:

License

Unlicense

Mastodon, it'll probably be extinct soon




鲜花

握手

雷人

路过

鸡蛋
该文章已有0人参与评论

请发表评论

全部评论

专题导读
上一篇:
stramargio/mastodon发布时间:2022-08-18
下一篇:
zoltan66/mastodon · GitHub发布时间:2022-08-18
热门推荐
阅读排行榜

扫描微信二维码

查看手机版网站

随时了解更新最新资讯

139-2527-9053

在线客服(服务时间 9:00~18:00)

在线QQ客服
地址:深圳市南山区西丽大学城创智工业园
电邮:jeky_zhao#qq.com
移动电话:139-2527-9053

Powered by 互联科技 X3.4© 2001-2213 极客世界.|Sitemap