Friday, February 14, 2020

Full Guide for using Bitnami Prometheus Operator Helm Chart with Additional Scrape Configuration

"The Prometheus Operator for Kubernetes provides easy monitoring definitions for Kubernetes services and deployment and management of Prometheus instances."
Bitnami kindly provided/maintained a Helm Chart of Prometheus Operator make it easy to start/deploy the Prometheus Operator in Kubernetes Cluster. Today, we will walk through the steps to get it running in your Kubernetes cluster.


  • Set up Helm and add Bitnami Helm Repo
    • You need have a Kubernetes cluster ready to deploy the Prometheus Operator on, I am using the docker-desktop cluster running on Windows 10. 
    • You need do "helm init" to install the Tiller to your running K8S cluster.
    • You need run "helm repo add bitnami https://charts.bitnami.com/bitnami" to add Bitnami Helm Repo.
  • Create the secret that contains your Additional Scrape Configuration
    • Create a local file named: additional-scrape-configs.yaml with content looks like:
    • Run the following command to make a secret out of the configuration file:
    • Apply the secret file to create the secret in your Kubernetes Cluster:
  • Install the bitnami/prometheus-operator Helm chart
  • Port forwarding for the prometheus to access it from your host machine
    • If everything went well, you should be able to access the prometheus from you host's web browser at http://localhost:9090
  • Last, you can start your prometheus instrumented app on your host to let the prometheus scrape some data.
    • You can go to the Targets on http://localhost:9090 to verify that the static target(s) configured Additional Scrape Configuration is UP.
If you don't have the correct named secret created or the format is malformed, the prometheus pod will not be started, and it is pretty hard to understand what is going on. Luckily you can watch the log from your svc/prometheus-operator-operator by using the following command to diagnose:
  • kubectl logs -f  svc/prometheus-operator-operator

Monday, February 10, 2020

Using FFmpeg streaming unix desktop to windows via UDP

"FFmpeg is a free and open-source project consisting of a vast software suite of libraries and programs for handling video, audio, and other multimedia files and streams."

FFmpeg is powerful, and sometime it is hard to figure out what exact the parameters to be used to achieve your goal with it. Today, we will teach you how to streaming Linux desktop to remote windows by using FFmpeg.

First, on your linux box start the FFmpeg streaming with the following command:

Note: The IP in the UDP url should be the remote windows IP. The FFmpeg will post the streaming data to that IP.

Then, on your remote windows box, start VLC 2.2.5 (an older version, be aware of that the current version 3.0.8, will not work, it just shows black screen when connect to the UDP streaming on windows), add a Network Stream:

udp://@0.0.0.01234

Voila, now you can view your Linux desktop on Windows now.

Wednesday, January 22, 2020

NATS multiple seed cluster

"NATS.io is a simple, secure and high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures."

There are some documents for NATS clustering, here and here.

Today, we would like to start a NATS cluster with two seed servers, the architecture look like: 


// To fill in the architecture diagram


Here are the steps for starting up the cluster:

  • Start up the two seed servers (seed1 and seed2):

  • Start three cluster servers (server A, B, C):


Voila! You get the cluster running, now it's time test it out:
  • Start subscriber and publisher:

Wednesday, October 23, 2019

MySQL docker volumes on Docker Desktop Windows 10

Using MySQL docker image is a quick and easy way to spin up an MySQL database for your application development and testing. However, if you are running on Windows Docker Desktop on Windows 10, if you do not have a correct volume setup, then the MySQL data volume will be embbed in the HyperV virual disk and is is very hard to reset/clean up. To mitigate the aforementioned problem, you can setup the correct volume on your host and share it with the Docker Desktop, this gives you full control for the MySQL data volume running in the Docker. Here are the steps:

  • create a local folder to hold the MySQL data volume:
    • mkdir c:\tmp\mysql
  • In your Docker Desktop via Settings -> Shared Drives mark your c: drive shared with Docker Desktop
  • Start your MySQL docker container:
    • docker run --net=mysql-network --name mysql -v c:/tmp/mysql:/var/lib/mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=***** -d mysql:latest
  • Start MySQL client docker container:
    • docker run -it --network mysql-network --rm mysql:latest mysql -hmysql -uroot -p*****
  • If you do want to blow away the MySQL data volume, now you can easily do it on your host machine.

Friday, October 18, 2019

How to use Intellij to debug Lagom application

When you are learning and running Lagom project, you may want to setup debug in your IDE - Intellij to be able to debug the Lagom application to get some insight on what is going on under the hood. Here is one way how you can setup the Intellij Remote debug to debug Lagom application:

  • First in your terminal run:
    • sbt -jvm-debug 5005
      • this will start the sbt shell with JDWP server on 5005
      • you can use IntelliJ Remote debug to attach the JVM
    • in sbt shell, run: 
      • runAll
  • From then on you should be able to setup break point and debugging on.

Thursday, October 17, 2019

Thingsboard install_dev_db HSQLDB spring.datasource.hikari.maximumPoolSize problem

Thingsboard is a fantastic open source IoT data gathering and visualization platform. I was trying to setup the local development mode for Thingsboard and always step on the mines, so I just want to document the problem and solution for install_dev_db. When you check out the Thingsboard source from github, you need to build the project on your computer:

  • mvn clean install -DskipTests
Then you try to run the application by using your IDE to run java main file: org.thingsboard.server.ThingsboardServerApplication, you will get the following error:
Schema-validation: missing table [admin_settings]

This is an indication that you don't have the database setup, for running in dev mode, we need to swith the DB to HSQLDB by commenting out the Postgres DB configuration and plug in the HSQLDB configuration found here.

When you try to run the install_dev_db.bat you may run into the following error:

Error creating bean with name 'baseTimeseriesService': Unsatisfied dependency expressed through field 'timeseriesDao'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jpaTimeseriesDao': Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'spring.datasource.hikari.maximumPoolSize' in value "${spring.datasource.hikari.maximumPoolSize}"

To solve the aforementioned error, you need add the following configuration in your thingsboard.yml under the spring.datasource:


hikari:
  maximumPoolSize: "${SPRING_DATASOURCE_MAXIMUM_POOL_SIZE:50}"
This will let you successfully setup the dev DB for thingsboard.

Tuesday, May 23, 2017

Setup your own Git server on windows running on NodeJS

As technology moving ahead, now days Git is the de facto version control system to used, if you would like to have a local private git server running on windows, here is an option running on NodeJS - git-server-windows, to setup up it is pretty easy:


  • Create a directory where you want to install the git-server-windows, cd to that directory, run 'npm install git-server-windows'
  • copy node_moduels/git-server-windows/*.cmd .
  • Create a directory: repos
  • cd to repos and create your repository, for example, myRepository; then cd to myRepository, run ' git init --bare'
  • In the top folder, create index.js and paste the following content:

    • var Git = require('git-server-windows');
      Git.server();
  • Run 'node index.js' to start the server
  •  Setup as windows services by using node-windows



Full Guide for using Bitnami Prometheus Operator Helm Chart with Additional Scrape Configuration

"The Prometheus Operator for Kubernetes provides easy monitoring definitions for Kubernetes services and deployment and management of...