OneStop is a data discovery system being built by CIRES researchers on a grant from the NOAA National Centers for Environmental Information. We welcome contributions from the community!
This project is maintained by cedardevs
Estimated Reading Time: 10 minutes
Reference the Quickstart Guide
The system uses Skaffold and Jib as a Gradle plugin to help with continuous development and deployment of Kubernetes applications.
How it works:
This means that Jib will monitor and catch any changes made to the source code and trigger a build. This saves a separate docker image and redeploys the application.
skaffold dev
Which will:
At this point, modifying a source file in a subproject will:
If you would rather not have gradle and skaffold watching for every change, you can skip the -t
option.
skaffold dev
to build and deploy your app every time your code changes,skaffold run
to build and deploy your app once, similar to a CI/CD pipelineThe registry api is exposed through ambassador via a LoadBalancer service. When using docker desktop, you can access them at:
http://localhost/onestop/api/registry # psi-registry
From there you can do things like upload test metadata to test the system. Please refer to the quickstart guide too.
The zookeeper, kafka, and registry deployments are configured to store their data in volumes allocated by
persistent volume claims. These claims remain when pods are removed and even when the deployments are deleted.
You can see the claims and their corresponding volumes them with kubectl
, like this:
kubectl get pvc,pv
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/data-kafka-sts-0 Bound pvc-e602da5b-5e25-11e8-9f48-080027a6b205 1Gi RWO standard 9d
persistentvolumeclaim/data-registry-0 Bound pvc-dc8c32dc-65c0-11e8-8652-080027a6b205 1Gi RWO standard 3h
persistentvolumeclaim/data-zookeeper-sts-0 Bound pvc-3860e2c5-5de4-11e8-9f48-080027a6b205 1Gi RWO standard 10d
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/pvc-3860e2c5-5de4-11e8-9f48-080027a6b205 1Gi RWO Delete Bound default/data-zookeeper-sts-0 standard 10d
persistentvolume/pvc-dc8c32dc-65c0-11e8-8652-080027a6b205 1Gi RWO Delete Bound default/data-registry-0 standard 3h
persistentvolume/pvc-e602da5b-5e25-11e8-9f48-080027a6b205 1Gi RWO Delete Bound default/data-kafka-sts-0 standard 9d
To remove the persistent data, e.g. to wipe everything out and start over, you just need to delete the persistent volume claims:
kubectl delete pvc data-registry-0
kubectl delete pvc data-kafka-sts-0
kubectl delete pvc data-zookeeper-sts-0
Or, to delete all volume claims in the current namespace:
kubectl delete pvc --all
From the root of this repo, you can run gradle task: for the entire project
./gradlew -t test bootJar shadowJar
for each subproject
./gradlew -t test `applicationName`:bootJar shadowJar
Which will: