{"componentChunkName":"component---src-pages-author-author-yaml-id-js","path":"/author/andy-yeung/","result":{"data":{"allMarkdownRemark":{"edges":[{"node":{"id":"4a994147-d7b5-56c2-ace6-b9d19e677ae3","html":"<p>Serverless computing has been gaining a lot of popularity in the last couple of years, and for a good reason. It saves developer's time, allowing them to <a href=\"https://www.loginradius.com/blog/engineering/learning-how-to-code/\">focus on writing code</a> rather than dealing with infrastructure. Deploying powerful and scalable applications has generally become much quicker and easier with serverless. However, there can be drawbacks as well, such as vendor lock-in and lack of control.</p>\n<p>This blog will briefly go over some of the most popular serverless computing options from major cloud providers. There are also plenty of great resources online explaining serverless architecture in detail, so definitely go check those out if you haven't already. Now, let's go over some of the popular choices for serverless computing.</p>\n<h2 id=\"popular-cloud-options\" style=\"position:relative;\"><a href=\"#popular-cloud-options\" aria-label=\"popular cloud options permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Popular Cloud Options</h2>\n<p>Almost all major cloud providers offer serverless computing such as AWS, Azure, Google Cloud, IBM Cloud, Alibaba Cloud, etc. In this blog, we'll only be going over a few.</p>\n<h3 id=\"1-aws-lambda\" style=\"position:relative;\"><a href=\"#1-aws-lambda\" aria-label=\"1 aws lambda permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>1. AWS Lambda</h3>\n<p>AWS Lambda was released in 2014 and has played a significant role in the rise of serverless computing.</p>\n<ul>\n<li>Lambda functions run using <a href=\"https://github.com/firecracker-microvm/firecracker\">AWS Firecracker</a>, an open-source virtualization project that focuses on securing and isolating functions while enabling the best performance possible.</li>\n<li>It supports plenty of runtimes: Node.js, Python, Ruby, Java, Go, .NET Core, and even custom runtimes for cases where the desired runtime is not natively supported.</li>\n<li>An excellent way to get support is through their official public <a href=\"https://forums.aws.amazon.com/forum.jspa?forumID=186\">forum</a>, where AWS employees can answer your questions.</li>\n<li><a href=\"https://www.loginradius.com/blog/engineering/a-journey-with-AWS/\">AWS Lambda offers</a> free usage of 1M requests per month and 400,000 GB-seconds of computing time per month.</li>\n</ul>\n<p>Recent notable updates include:</p>\n<ul>\n<li><a href=\"https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/\">Dec 2020</a>: Function's can be defined as container images up to 10GB. Previously a function's code had to be written through the UI code editor or uploaded as zip archives.</li>\n<li><a href=\"https://aws.amazon.com/blogs/aws/new-for-aws-lambda-functions-with-up-to-10-gb-of-memory-and-6-vcpus/\">Dec 2020</a>: Function's memory can be configurable up to 10GB. Previously, the memory limit was 3GB, so this is an increase of over three times. Since Lambda automatically allocates CPU in proportion to the memory limit, there can be up to 6 vCPUs.</li>\n<li><a href=\"https://aws.amazon.com/blogs/aws/new-for-aws-lambda-1ms-billing-granularity-adds-cost-savings/\">Dec 2020</a>: For pricing calculation, function execution duration, rounded up to the nearest 1ms. Previously, duration was nearest to 100ms, which means lower cost for functions executing under 100ms.</li>\n<li><a href=\"https://aws.amazon.com/blogs/aws/new-a-shared-file-system-for-your-lambda-functions/\">June 2020</a>: Share a file system between Lambda functions via Amazon EFS.</li>\n</ul>\n<h3 id=\"2-azure-functions\" style=\"position:relative;\"><a href=\"#2-azure-functions\" aria-label=\"2 azure functions permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>2. Azure Functions</h3>\n<p>Launched in 2016:</p>\n<ul>\n<li>Part of the open-source initiative, which started back when Microsoft purchased GitHub in 2018. This includes the Azure function runtime, core tools, portal, and more. Check them out <a href=\"https://github.com/Azure/Azure-Functions\">here</a>!</li>\n<li>Runtime support includes .NET Framework, .NET Core, Node.js, Java, Python, and more.</li>\n<li>Offers free usage of 1M requests per month and 400,000 GB-seconds of computing time per month.</li>\n</ul>\n<p>Recent notable updates include:</p>\n<ul>\n<li><a href=\"https://azure.microsoft.com/en-us/updates/azure-functions-custom-handlers-are-now-generally-available/\">Dec 2020</a>: Custom handlers are now supported, allowing creating functions in any runtime.</li>\n<li><a href=\"https://azure.microsoft.com/en-gb/updates?id=rabbitmq-extension-for-windows-and-linux-is-now-generally-available\">Dec 2020</a>: Functions can now send and receive messages to and from RabbitMQ.</li>\n</ul>\n<h3 id=\"3-google-cloud-functions\" style=\"position:relative;\"><a href=\"#3-google-cloud-functions\" aria-label=\"3 google cloud functions permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>3. Google Cloud Functions</h3>\n<p>Launched in 2017:</p>\n<ul>\n<li>Run with the help of <a href=\"https://github.com/google/gvisor\">gVisor</a> for sandboxing containers. Since containers were never designed to be secure sandboxes, gVisor is a tool Google built to protect against potential vulnerabilities allowing container escapes and others.</li>\n<li>Runtime support includes Node.js, Go, Ruby, Java, Python, .NET Core.</li>\n<li>Offers free usage of 2M requests per month and 400,000 GB-seconds of computing time per month. Allows 1M more free requests per month compared to AWS and Azure!</li>\n</ul>\n<p>Recent notable updates include:</p>\n<ul>\n<li><a href=\"https://cloud.google.com/blog/products/application-development/ruby-comes-to-cloud-functions\">Jan 2021</a>: Support Ruby runtime.</li>\n<li><a href=\"https://cloud.google.com/blog/products/application-development/introducing-net-google-cloud-functions\">Nov 2020</a>: Support .NET Core runtime.</li>\n</ul>\n<h3 id=\"4-cloudflare-workers\" style=\"position:relative;\"><a href=\"#4-cloudflare-workers\" aria-label=\"4 cloudflare workers permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>4. CloudFlare Workers</h3>\n<p>Launched in 2017:</p>\n<ul>\n<li>Takes advantage of Cloudflare's edge locations for excellent performance.</li>\n<li>Run using the V8 JavaScript engine, allowing for much less overhead than using VMs or containers.</li>\n<li>V8 engine allows Cloudflare Workers to run JavaScript. V8 engine includes languages that can compile to WebAssembly, such as Rust, C, Cobol. Also, languages that can compile to JavaScript, such as Python, Kotlin, Perl.</li>\n<li>Offers free usage of 100,000 requests per day and a 128MB worker memory limit. Details on all their free plan restrictions <a href=\"https://developers.cloudflare.com/workers/platform/limits#worker-limits\">here</a>.</li>\n</ul>\n<p>Recent notable updates include:</p>\n<ul>\n<li><a href=\"https://blog.cloudflare.com/introducing-workers-durable-objects/\">Sep 2020</a>: Maintain state with durable objects.</li>\n<li><a href=\"https://blog.cloudflare.com/introducing-cron-triggers-for-cloudflare-workers/\">Sep 2020</a>: Use a scheduler to trigger workers on a timed interval.</li>\n</ul>\n<h2 id=\"conclusion\" style=\"position:relative;\"><a href=\"#conclusion\" aria-label=\"conclusion permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Conclusion</h2>\n<p>There are a lot of great options worth considering when looking for serverless computing from a cloud provider. This was a very brief introduction to some of the most popular choices, so definitely check out each provider's official documentation for more detail!</p>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n</style>","frontmatter":{"title":"Top 4 Serverless Computing Platforms in 2021","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"March 04, 2021","updated_date":null,"tags":["Serverless"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.5037593984962405,"src":"/static/5c35cac6bac6aa7487ccce9ebb7d8d94/ee604/cover.png","srcSet":"/static/5c35cac6bac6aa7487ccce9ebb7d8d94/69585/cover.png 200w,\n/static/5c35cac6bac6aa7487ccce9ebb7d8d94/497c6/cover.png 400w,\n/static/5c35cac6bac6aa7487ccce9ebb7d8d94/ee604/cover.png 800w,\n/static/5c35cac6bac6aa7487ccce9ebb7d8d94/f3583/cover.png 1200w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/serverless-overview/"}}},{"node":{"id":"7f4aaef1-b2c3-5939-b72b-07b24a928765","html":"<p>This blog will help you get started on deploying your REST API in Kubernetes. First, we'll set up a local Kubernetes cluster, then create a <a href=\"https://www.loginradius.com/blog/engineering/what-is-an-api/\">simple API</a> to deploy.</p>\n<p>There are already a lot of <a href=\"https://www.quora.com/What-are-the-best-resources-to-learn-Kubernetes\">free resources available</a> explaining basic Kubernetes concepts, so go check those out first if you haven't already. This blog is intended for beginners but assumes you already have a <a href=\"https://www.loginradius.com/blog/engineering/understanding-kubernetes/\">basic understanding of Kubernetes</a> and Docker concepts.</p>\n<h2 id=\"1-set-up-local-kubernetes\" style=\"position:relative;\"><a href=\"#1-set-up-local-kubernetes\" aria-label=\"1 set up local kubernetes permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>1. Set Up Local Kubernetes</h2>\n<p>There's a couple options for running Kubernetes locally, with the most popular ones including <a href=\"https://github.com/kubernetes/minikube\">minikube</a>, <a href=\"https://github.com/k3s-io/k3s\">k3s</a>, <a href=\"https://github.com/kubernetes-sigs/kind\">kind</a>, <a href=\"https://github.com/ubuntu/microk8s\">microk8s</a>. In this guide, any of these will work, but we will be using k3s because of the lightweight installation.</p>\n<p>Install <a href=\"https://github.com/rancher/k3d\">k3d</a>, which is a utility for running k3s. k3s will be running in Docker, so make sure you have that installed as well. We used k3d v4.0 in this blog.</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"0\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">curl -s https://raw.githubusercontent.com/rancher/k3d/main/install.sh | bash</span></code></pre>\n<p>Set up a cluster named test:</p>\n<ul>\n<li>The port flag is for mapping port 80 from our machine to port 80 on the k3s load balancer. This is needed later when we use ingress.</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"1\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">k3d cluster create test -p &quot;80:80@loadbalancer&quot;</span></code></pre>\n<p>Optionally, check that your kubeconfig got updated and the current context is correct:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"2\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl config view</span>\n<span class=\"grvsc-line\">kubectl config current-context</span></code></pre>\n<p>Optionally, confirm that k3s is running in Docker. There should be two containers up, one for k3s and the other for load balancing:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"3\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">docker ps</span></code></pre>\n<p>Make sure that all the pods are running. If they are stuck in pending status, it may be that there is not enough disk space on your machine. You can get more information by using the describe command:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"4\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl get pods -A</span>\n<span class=\"grvsc-line\">kubectl describe pods -A</span></code></pre>\n<p>There's a lot of kubectl commands you can try, so I recommend checking out the list of resources and being aware of their short names:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"5\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl api-resources</span></code></pre>\n<h2 id=\"2-create-a-simple-api\" style=\"position:relative;\"><a href=\"#2-create-a-simple-api\" aria-label=\"2 create a simple api permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>2. Create a Simple API</h2>\n<p>We will create a simple API using Express.js.</p>\n<p>Set up the project:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"6\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">mkdir my-backend-api && cd my-backend-api</span>\n<span class=\"grvsc-line\">touch server.js</span>\n<span class=\"grvsc-line\">npm init</span>\n<span class=\"grvsc-line\">npm i express --save</span></code></pre>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"7\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// server.js</span>\n<span class=\"grvsc-line\">const express = require(&quot;express&quot;);</span>\n<span class=\"grvsc-line\">const app = express();</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">app.get(&quot;/user/:id&quot;, (req, res) =&gt; {</span>\n<span class=\"grvsc-line\">  const id = req.params.id;</span>\n<span class=\"grvsc-line\">  res.json({</span>\n<span class=\"grvsc-line\">    id,</span>\n<span class=\"grvsc-line\">    name: `John Doe #${id}`</span>\n<span class=\"grvsc-line\">  });</span>\n<span class=\"grvsc-line\">});</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">app.listen(80, () =&gt; {</span>\n<span class=\"grvsc-line\">  console.log(&quot;Server running on port 80&quot;);</span>\n<span class=\"grvsc-line\">});</span></code></pre>\n<p>Optionally, you can try running it if you have Node.js installed and test the endpoint /user/{id} with curl:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"8\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">node server.js</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">// request:</span>\n<span class=\"grvsc-line\">curl http://localhost:80/user/123</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">// response: {&quot;id&quot;:&quot;123&quot;,&quot;name&quot;:&quot;John Doe #123&quot;}</span></code></pre>\n<p>Next, add a Dockerfile and .dockerignore:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"9\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// Dockerfile</span>\n<span class=\"grvsc-line\">FROM node:12</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">WORKDIR /usr/src/app</span>\n<span class=\"grvsc-line\">COPY package*.json ./</span>\n<span class=\"grvsc-line\">RUN npm i</span>\n<span class=\"grvsc-line\">COPY . .</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">EXPOSE 80</span>\n<span class=\"grvsc-line\">CMD [&quot;node&quot;, &quot;server.js&quot;]</span></code></pre>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"10\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// .dockerignore</span>\n<span class=\"grvsc-line\">node_modules</span></code></pre>\n<p>Then, build the image and push it to the Docker Hub registry:</p>\n<ul>\n<li>If you want to skip this step, you can use the existing image <a href=\"https://hub.docker.com/r/andyy5/my-backend-api\">here</a>.</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"11\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">docker build -t &lt;YOUR_DOCKER_ID&gt;/my-backend-api .</span>\n<span class=\"grvsc-line\">docker push &lt;YOUR_DOCKER_ID&gt;/my-backend-api</span></code></pre>\n<h2 id=\"3-deploy\" style=\"position:relative;\"><a href=\"#3-deploy\" aria-label=\"3 deploy permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>3. Deploy</h2>\n<p>Now, we deploy the image to our local Kubernetes cluster. We use the default namespace.</p>\n<p>Create a deployment:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"12\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl create deploy my-backend-api --image=andyy5/my-backend-api</span></code></pre>\n<ul>\n<li>Alternatively, create a deployment with a YAML file:</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"13\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl create -f deployment.yaml</span></code></pre>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"14\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// deployment.yaml</span>\n<span class=\"grvsc-line\">apiVersion: apps/v1</span>\n<span class=\"grvsc-line\">kind: Deployment</span>\n<span class=\"grvsc-line\">metadata:</span>\n<span class=\"grvsc-line\">  name: my-backend-api</span>\n<span class=\"grvsc-line\">  labels:</span>\n<span class=\"grvsc-line\">    app: my-backend-api</span>\n<span class=\"grvsc-line\">spec:</span>\n<span class=\"grvsc-line\">  replicas: 1</span>\n<span class=\"grvsc-line\">  selector:</span>\n<span class=\"grvsc-line\">    matchLabels:</span>\n<span class=\"grvsc-line\">      app: my-backend-api</span>\n<span class=\"grvsc-line\">  template:</span>\n<span class=\"grvsc-line\">    metadata:</span>\n<span class=\"grvsc-line\">      labels:</span>\n<span class=\"grvsc-line\">        app: my-backend-api</span>\n<span class=\"grvsc-line\">    spec:</span>\n<span class=\"grvsc-line\">      containers:</span>\n<span class=\"grvsc-line\">      - name: my-backend-api</span>\n<span class=\"grvsc-line\">        image: andyy5/my-backend-api</span></code></pre>\n<p>Create a service:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"15\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl expose deploy my-backend-api --type=ClusterIP --port=80</span></code></pre>\n<ul>\n<li>Alternatively, create a service with a YAML file:</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"16\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl create -f service.yaml</span></code></pre>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"17\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// service.yaml</span>\n<span class=\"grvsc-line\">apiVersion: v1</span>\n<span class=\"grvsc-line\">kind: Service</span>\n<span class=\"grvsc-line\">metadata:</span>\n<span class=\"grvsc-line\">  name: my-backend-api</span>\n<span class=\"grvsc-line\">  labels:</span>\n<span class=\"grvsc-line\">    app: my-backend-api</span>\n<span class=\"grvsc-line\">spec:</span>\n<span class=\"grvsc-line\">  type: ClusterIP</span>\n<span class=\"grvsc-line\">  ports:</span>\n<span class=\"grvsc-line\">  - port: 80</span>\n<span class=\"grvsc-line\">    protocol: TCP</span>\n<span class=\"grvsc-line\">    targetPort: 80</span>\n<span class=\"grvsc-line\">  selector:</span>\n<span class=\"grvsc-line\">    app: my-backend-api</span></code></pre>\n<p>Check that everything was created and the pod is running:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"18\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl get deploy -A</span>\n<span class=\"grvsc-line\">kubectl get svc -A</span>\n<span class=\"grvsc-line\">kubectl get pods -A</span></code></pre>\n<p>Once the pod is running, the API is accessible within the cluster only. One quick way to verify the deployment from our localhost is by doing port forwarding:</p>\n<ul>\n<li>Replace the pod name below with the one in your cluster</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"19\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl port-forward my-backend-api-84bb9d79fc-m9ddn 3000:80</span></code></pre>\n<ul>\n<li>Now, you can send a curl request from your machine</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"20\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">curl http://localhost:3000/user/123</span></code></pre>\n<p>To correctly manage external access to the services in a cluster, we need to use ingress. Close the port-forwarding and let's expose our API by creating an ingress resource.</p>\n<ul>\n<li>An ingress controller is also required, but k3d by default deploys the cluster with a Traefik ingress controller (listening on port 80).</li>\n<li>Recall that when we created our cluster, we set a port flag with the value \"80:80@loadbalancer\". If you missed this part, go back and create your cluster again.</li>\n</ul>\n<p>Create an Ingress resource with the following YAML file:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"21\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">kubectl create -f ingress.yaml</span>\n<span class=\"grvsc-line\">kubectl get ing -A</span></code></pre>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"22\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// ingress.yaml</span>\n<span class=\"grvsc-line\">apiVersion: networking.k8s.io/v1</span>\n<span class=\"grvsc-line\">kind: Ingress</span>\n<span class=\"grvsc-line\">metadata:</span>\n<span class=\"grvsc-line\">  name: my-backend-api</span>\n<span class=\"grvsc-line\">  annotations:</span>\n<span class=\"grvsc-line\">    ingress.kubernetes.io/ssl-redirect: &quot;false&quot;</span>\n<span class=\"grvsc-line\">spec:</span>\n<span class=\"grvsc-line\">  rules:</span>\n<span class=\"grvsc-line\">  - http:</span>\n<span class=\"grvsc-line\">      paths:</span>\n<span class=\"grvsc-line\">      - path: /user/</span>\n<span class=\"grvsc-line\">        pathType: Prefix</span>\n<span class=\"grvsc-line\">        backend:</span>\n<span class=\"grvsc-line\">          service:</span>\n<span class=\"grvsc-line\">            name: my-backend-api</span>\n<span class=\"grvsc-line\">            port:</span>\n<span class=\"grvsc-line\">              number: 80</span></code></pre>\n<ul>\n<li>Now try it out!</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"23\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">curl http://localhost:80/user/123</span></code></pre>\n<p>If you want to learn more on how to deploy using a managed Kubernetes service in the cloud, such as Google Kubernetes Engine, then check out the excellent guides on the <a href=\"https://kubernetes.io/docs/tutorials/stateless-application/expose-external-ip-address/\">official Kubernetes docs</a>.</p>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n  .dark-default-dark {\n    background-color: #1E1E1E;\n    color: #D4D4D4;\n  }\n</style>","frontmatter":{"title":"How to Deploy a REST API in Kubernetes","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"February 03, 2021","updated_date":null,"tags":["Kubernetes"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.492537313432836,"src":"/static/efa8ecb370a0a94f380c24981ede2913/ee604/cover.png","srcSet":"/static/efa8ecb370a0a94f380c24981ede2913/69585/cover.png 200w,\n/static/efa8ecb370a0a94f380c24981ede2913/497c6/cover.png 400w,\n/static/efa8ecb370a0a94f380c24981ede2913/ee604/cover.png 800w,\n/static/efa8ecb370a0a94f380c24981ede2913/f3583/cover.png 1200w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/rest-api-kubernetes/"}}},{"node":{"id":"4836456a-d38a-563c-babe-3f0a5b173c32","html":"<p>Let's walk through how to build and push Docker images programmatically using Go. To do this, we need to talk to the Docker daemon via the <a href=\"https://docs.docker.com/engine/api/\">Docker Engine API</a>. This is similar to how the Docker CLI works, but instead of entering commands through a CLI, we'll be writing code with Docker's Go SDK.</p>\n<p>At the time of writing, the official Docker Go SDK <a href=\"https://docs.docker.com/engine/api/sdk/examples/\">docs</a> provide great examples of running basic Docker commands with Go. However, it's missing examples on building and pushing Docker images, so we'll go over those in this blog.</p>\n<p>Before we begin, this blog assumes you have a working knowledge of Docker and Go. We'll go over the following:</p>\n<ul>\n<li>Building an image from local source code</li>\n<li>Pushing an image to a remote registry</li>\n</ul>\n<h2 id=\"environment-setup\" style=\"position:relative;\"><a href=\"#environment-setup\" aria-label=\"environment setup permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Environment Setup</h2>\n<p>First, we need to set up the environment. Create a project and include the app we want to containerize:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"0\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">mkdir docker-go-tutorial && cd docker-go-tutorial && mkdir node-hello</span></code></pre>\n<p>We'll add a simple Node.js app:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"1\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// node-hello/app.js</span>\n<span class=\"grvsc-line\">console.log(&quot;Hello From LoginRadius&quot;);</span></code></pre>\n<p>with the Dockerfile:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"2\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// node-hello/Dockerfile</span>\n<span class=\"grvsc-line\">FROM node:12</span>\n<span class=\"grvsc-line\">WORKDIR /src</span>\n<span class=\"grvsc-line\">COPY . .</span>\n<span class=\"grvsc-line\">CMD [ &quot;node&quot;, &quot;app.js&quot; ]</span></code></pre>\n<p>Next, install the <a href=\"https://docs.docker.com/engine/api/sdk/\">Go SDK</a>. These are the Docker related imports we will be using:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"3\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">&quot;github.com/docker/docker/api/types&quot;</span>\n<span class=\"grvsc-line\">&quot;github.com/docker/docker/client&quot;</span>\n<span class=\"grvsc-line\">&quot;github.com/docker/docker/pkg/archive&quot;</span></code></pre>\n<h2 id=\"build-docker-image\" style=\"position:relative;\"><a href=\"#build-docker-image\" aria-label=\"build docker image permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Build Docker Image</h2>\n<p>One way to build a Docker image from our local files is to compress those files into a tar archive first.</p>\n<p>We use the archive package provided by Docker:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"4\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">&quot;github.com/docker/docker/pkg/archive&quot;</span></code></pre>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"5\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">tar, err := archive.TarWithOptions(&quot;node-hello/&quot;, &archive.TarOptions{})</span>\n<span class=\"grvsc-line\">if err != nil {</span>\n<span class=\"grvsc-line\">    return err</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>Now, we can call the ImageBuild function using the Go SDK:</p>\n<ul>\n<li>\n<p>Note that the image tag includes our Docker registry user ID, so we can push this image to our registry later.</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"6\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">opts := types.ImageBuildOptions{</span>\n<span class=\"grvsc-line\">Dockerfile:  &quot;Dockerfile&quot;,</span>\n<span class=\"grvsc-line\">Tags:        []string{dockerRegistryUserID + &quot;/node-hello&quot;},</span>\n<span class=\"grvsc-line\">Remove:      true,</span>\n<span class=\"grvsc-line\">}</span>\n<span class=\"grvsc-line\">res, err := dockerClient.ImageBuild(ctx, tar, opts)</span>\n<span class=\"grvsc-line\">if err != nil {</span>\n<span class=\"grvsc-line\">return err</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n</li>\n</ul>\n<p>To print the response, we use a scanner to go through line by line:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"7\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">scanner := bufio.NewScanner(res.Body)</span>\n<span class=\"grvsc-line\">for scanner.Scan() {</span>\n<span class=\"grvsc-line\">    lastLine = scanner.Text()</span>\n<span class=\"grvsc-line\">    fmt.Println(scanner.Text())</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>This prints the following:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"8\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">{&quot;stream&quot;:&quot;Step 1/4 : FROM node:12&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot; ---\\u003e e4f1e16b3633\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;Step 2/4 : WORKDIR /src&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot; ---\\u003e Using cache\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot; ---\\u003e b298b8519669\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;Step 3/4 : COPY . .&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot; ---\\u003e Using cache\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot; ---\\u003e 1ff6a87e79d9\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;Step 4/4 : CMD [ \\&quot;node\\&quot;, \\&quot;app.js\\&quot; ]&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot; ---\\u003e Using cache\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot; ---\\u003e 6ca44f72b68d\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;aux&quot;:{&quot;ID&quot;:&quot;sha256:238a923459uf28h80103eb089804a2ff2c1f68f8c&quot;}}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;Successfully built 6ca44f72b68d\\n&quot;}</span>\n<span class=\"grvsc-line\">{&quot;stream&quot;:&quot;Successfully tagged lrblake/node-hello:latest\\n&quot;}</span></code></pre>\n<p>The last step would be checking the response for errors, so if something went wrong during the build, we could handle it.</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"9\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">errLine := &ErrorLine{}</span>\n<span class=\"grvsc-line\">json.Unmarshal([]byte(lastLine), errLine)</span>\n<span class=\"grvsc-line\">if errLine.Error != &quot;&quot; {</span>\n<span class=\"grvsc-line\">    return errors.New(errLine.Error)</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>For example, the following error can occur during build:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"10\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">{&quot;errorDetail&quot;:{&quot;message&quot;:&quot;COPY failed: stat /var/lib/docker/tmp/docker-builder887191115/z: no such file or directory&quot;},&quot;error&quot;:&quot;COPY failed: stat /var/lib/docker/tmp/docker-builder887191115/z: no such file or directory&quot;}</span></code></pre>\n<p>All together, the file looks like this:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"go\" data-index=\"11\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk4\">package</span><span class=\"mtk1\"> main</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">import</span><span class=\"mtk1\"> (</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;bufio&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;context&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;encoding/json&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;errors&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;fmt&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;io&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;time&quot;</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;github.com/docker/docker/api/types&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;github.com/docker/docker/client&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk8\">&quot;github.com/docker/docker/pkg/archive&quot;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">)</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">var</span><span class=\"mtk1\"> </span><span class=\"mtk12\">dockerRegistryUserID</span><span class=\"mtk1\"> = </span><span class=\"mtk8\">&quot;&quot;</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">type</span><span class=\"mtk1\"> </span><span class=\"mtk10\">ErrorLine</span><span class=\"mtk1\"> </span><span class=\"mtk4\">struct</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\tError       </span><span class=\"mtk10\">string</span><span class=\"mtk1\">      </span><span class=\"mtk8\">`json:&quot;error&quot;`</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\tErrorDetail ErrorDetail </span><span class=\"mtk8\">`json:&quot;errorDetail&quot;`</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">type</span><span class=\"mtk1\"> </span><span class=\"mtk10\">ErrorDetail</span><span class=\"mtk1\"> </span><span class=\"mtk4\">struct</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\tMessage </span><span class=\"mtk10\">string</span><span class=\"mtk1\"> </span><span class=\"mtk8\">`json:&quot;message&quot;`</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">func</span><span class=\"mtk1\"> </span><span class=\"mtk11\">main</span><span class=\"mtk1\">() {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">cli</span><span class=\"mtk1\">, </span><span class=\"mtk12\">err</span><span class=\"mtk1\"> := client.</span><span class=\"mtk11\">NewClientWithOpts</span><span class=\"mtk1\">(client.FromEnv, client.</span><span class=\"mtk11\">WithAPIVersionNegotiation</span><span class=\"mtk1\">())</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">if</span><span class=\"mtk1\"> err != </span><span class=\"mtk4\">nil</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\tfmt.</span><span class=\"mtk11\">Println</span><span class=\"mtk1\">(err.</span><span class=\"mtk11\">Error</span><span class=\"mtk1\">())</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk15\">return</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">err</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">imageBuild</span><span class=\"mtk1\">(cli)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">if</span><span class=\"mtk1\"> err != </span><span class=\"mtk4\">nil</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\tfmt.</span><span class=\"mtk11\">Println</span><span class=\"mtk1\">(err.</span><span class=\"mtk11\">Error</span><span class=\"mtk1\">())</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk15\">return</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">func</span><span class=\"mtk1\"> </span><span class=\"mtk11\">imageBuild</span><span class=\"mtk1\">(dockerClient *client.Client) </span><span class=\"mtk10\">error</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">ctx</span><span class=\"mtk1\">, </span><span class=\"mtk12\">cancel</span><span class=\"mtk1\"> := context.</span><span class=\"mtk11\">WithTimeout</span><span class=\"mtk1\">(context.</span><span class=\"mtk11\">Background</span><span class=\"mtk1\">(), time.Second*</span><span class=\"mtk7\">120</span><span class=\"mtk1\">)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">defer</span><span class=\"mtk1\"> </span><span class=\"mtk11\">cancel</span><span class=\"mtk1\">()</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">tar</span><span class=\"mtk1\">, </span><span class=\"mtk12\">err</span><span class=\"mtk1\"> := archive.</span><span class=\"mtk11\">TarWithOptions</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&quot;node-hello/&quot;</span><span class=\"mtk1\">, &archive.TarOptions{})</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">if</span><span class=\"mtk1\"> err != </span><span class=\"mtk4\">nil</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk15\">return</span><span class=\"mtk1\"> err</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">opts</span><span class=\"mtk1\"> := types.ImageBuildOptions{</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\tDockerfile: </span><span class=\"mtk8\">&quot;Dockerfile&quot;</span><span class=\"mtk1\">,</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\tTags:       []</span><span class=\"mtk10\">string</span><span class=\"mtk1\">{dockerRegistryUserID + </span><span class=\"mtk8\">&quot;/node-hello&quot;</span><span class=\"mtk1\">},</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\tRemove:     </span><span class=\"mtk4\">true</span><span class=\"mtk1\">,</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">res</span><span class=\"mtk1\">, </span><span class=\"mtk12\">err</span><span class=\"mtk1\"> := dockerClient.</span><span class=\"mtk11\">ImageBuild</span><span class=\"mtk1\">(ctx, tar, opts)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">if</span><span class=\"mtk1\"> err != </span><span class=\"mtk4\">nil</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk15\">return</span><span class=\"mtk1\"> err</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">defer</span><span class=\"mtk1\"> res.Body.</span><span class=\"mtk11\">Close</span><span class=\"mtk1\">()</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">err</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">print</span><span class=\"mtk1\">(res.Body)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">if</span><span class=\"mtk1\"> err != </span><span class=\"mtk4\">nil</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk15\">return</span><span class=\"mtk1\"> err</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">return</span><span class=\"mtk1\"> </span><span class=\"mtk4\">nil</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">func</span><span class=\"mtk1\"> </span><span class=\"mtk11\">print</span><span class=\"mtk1\">(rd io.Reader) </span><span class=\"mtk10\">error</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk4\">var</span><span class=\"mtk1\"> </span><span class=\"mtk12\">lastLine</span><span class=\"mtk1\"> </span><span class=\"mtk10\">string</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">scanner</span><span class=\"mtk1\"> := bufio.</span><span class=\"mtk11\">NewScanner</span><span class=\"mtk1\">(rd)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">for</span><span class=\"mtk1\"> scanner.</span><span class=\"mtk11\">Scan</span><span class=\"mtk1\">() {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk12\">lastLine</span><span class=\"mtk1\"> = scanner.</span><span class=\"mtk11\">Text</span><span class=\"mtk1\">()</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\tfmt.</span><span class=\"mtk11\">Println</span><span class=\"mtk1\">(scanner.</span><span class=\"mtk11\">Text</span><span class=\"mtk1\">())</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk12\">errLine</span><span class=\"mtk1\"> := &ErrorLine{}</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\tjson.</span><span class=\"mtk11\">Unmarshal</span><span class=\"mtk1\">([]</span><span class=\"mtk11\">byte</span><span class=\"mtk1\">(lastLine), errLine)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">if</span><span class=\"mtk1\"> errLine.Error != </span><span class=\"mtk8\">&quot;&quot;</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk15\">return</span><span class=\"mtk1\"> errors.</span><span class=\"mtk11\">New</span><span class=\"mtk1\">(errLine.Error)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">if</span><span class=\"mtk1\"> </span><span class=\"mtk12\">err</span><span class=\"mtk1\"> := scanner.</span><span class=\"mtk11\">Err</span><span class=\"mtk1\">(); err != </span><span class=\"mtk4\">nil</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t\t</span><span class=\"mtk15\">return</span><span class=\"mtk1\"> err</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t}</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">\t</span><span class=\"mtk15\">return</span><span class=\"mtk1\"> </span><span class=\"mtk4\">nil</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}</span></span></code></pre>\n<p>The equivalent Docker CLI command would be:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"12\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">docker build -t &lt;dockerRegistryUserID&gt;/node-hello .</span></code></pre>\n<h2 id=\"push-docker-image\" style=\"position:relative;\"><a href=\"#push-docker-image\" aria-label=\"push docker image permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Push Docker Image</h2>\n<p>We'll push the Docker image we created to Docker Hub. But, we need to authenticate with Docker Hub by providing credentials encoded in base64.</p>\n<ul>\n<li>In practice, don't hardcode your credentials in your source code.</li>\n<li>\n<p>If you don't want to use your Docker Hub password, you can set up an access token and provide that in the Password field instead.</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"13\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">var authConfig = types.AuthConfig{</span>\n<span class=\"grvsc-line\">Username:      &quot;Your Docker Hub Username&quot;,</span>\n<span class=\"grvsc-line\">Password:      &quot;Your Docker Hub Password or Access Token&quot;,</span>\n<span class=\"grvsc-line\">ServerAddress: &quot;https://index.docker.io/v1/&quot;,</span>\n<span class=\"grvsc-line\">}</span>\n<span class=\"grvsc-line\">authConfigBytes, _ := json.Marshal(authConfig)</span>\n<span class=\"grvsc-line\">authConfigEncoded := base64.URLEncoding.EncodeToString(authConfigBytes)</span></code></pre>\n</li>\n</ul>\n<p>Now, call the ImagePush function in the Go SDK, along with your encoded credentials:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"14\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">opts := types.ImagePushOptions{RegistryAuth: authConfigEncoded}</span>\n<span class=\"grvsc-line\">rd, err := dockerClient.ImagePush(ctx, dockerRegistryUserID + &quot;/node-hello&quot;, opts)</span></code></pre>\n<p>Together, this looks like:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"15\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">func imagePush(dockerClient *client.Client) error {</span>\n<span class=\"grvsc-line\">\tctx, cancel := context.WithTimeout(context.Background(), time.Second*120)</span>\n<span class=\"grvsc-line\">\tdefer cancel()</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">\tauthConfigBytes, _ := json.Marshal(authConfig)</span>\n<span class=\"grvsc-line\">\tauthConfigEncoded := base64.URLEncoding.EncodeToString(authConfigBytes)</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">\ttag := dockerRegistryUserID + &quot;/node-hello&quot;</span>\n<span class=\"grvsc-line\">\topts := types.ImagePushOptions{RegistryAuth: authConfigEncoded}</span>\n<span class=\"grvsc-line\">\trd, err := dockerClient.ImagePush(ctx, tag, opts)</span>\n<span class=\"grvsc-line\">\tif err != nil {</span>\n<span class=\"grvsc-line\">\t\treturn err</span>\n<span class=\"grvsc-line\">\t}</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">\tdefer rd.Close()</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">\terr = print(rd)</span>\n<span class=\"grvsc-line\">\tif err != nil {</span>\n<span class=\"grvsc-line\">\t\treturn err</span>\n<span class=\"grvsc-line\">\t}</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">\treturn nil</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>The equivalent Docker CLI command (after docker login) would be:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"16\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">docker push &lt;dockerRegistryUserID&gt;/node-hello</span></code></pre>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n  .dark-default-dark {\n    background-color: #1E1E1E;\n    color: #D4D4D4;\n  }\n  .dark-default-dark .mtk4 { color: #569CD6; }\n  .dark-default-dark .mtk1 { color: #D4D4D4; }\n  .dark-default-dark .mtk8 { color: #CE9178; }\n  .dark-default-dark .mtk12 { color: #9CDCFE; }\n  .dark-default-dark .mtk10 { color: #4EC9B0; }\n  .dark-default-dark .mtk11 { color: #DCDCAA; }\n  .dark-default-dark .mtk15 { color: #C586C0; }\n  .dark-default-dark .mtk7 { color: #B5CEA8; }\n</style>","frontmatter":{"title":"Build and Push Docker Images with Go","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"December 08, 2020","updated_date":null,"tags":["Docker","Go"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.680672268907563,"src":"/static/f7f513d2d6c5460c9f71278fcb98f766/ee604/cover.png","srcSet":"/static/f7f513d2d6c5460c9f71278fcb98f766/69585/cover.png 200w,\n/static/f7f513d2d6c5460c9f71278fcb98f766/497c6/cover.png 400w,\n/static/f7f513d2d6c5460c9f71278fcb98f766/ee604/cover.png 800w,\n/static/f7f513d2d6c5460c9f71278fcb98f766/31987/cover.png 1000w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/build-push-docker-images-golang/"}}},{"node":{"id":"92200a97-74be-5423-a4f1-a30f7eb0bea6","html":"<h2 id=\"what-is-pgp\" style=\"position:relative;\"><a href=\"#what-is-pgp\" aria-label=\"what is pgp permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>What is PGP?</h2>\n<p>PGP (Pretty Good Privacy) is a cryptographic process used to encrypt and decrypt information. It combines concepts from symmetric and asymmetric key encryption, maintaining some of the best security and usability aspects of both.</p>\n<p>One way PGP can be used is to protect the confidentiality of information. Once the information is encrypted, nobody will be able to decrypt it unless they have the right key. In practice, PGP is commonly used in sending and receiving emails, sharing information on the Dark Web, and others. This is because both on and off the Internet, there are ways to intercept information being sent, making encryption using PGP or similar critical.</p>\n<p>On a high-level the process between a sender and receiver looks like this:</p>\n<ol>\n<li>The recipient generates public and private keys.</li>\n<li>The recipient sends its public key to the sender.</li>\n<li>The sender encrypts the message using the given public key.</li>\n<li>The sender sends the encrypted message to the recipient.</li>\n<li>The recipient decrypts the message using its private key.</li>\n</ol>\n<h2 id=\"pgp-examples-in-nodejs\" style=\"position:relative;\"><a href=\"#pgp-examples-in-nodejs\" aria-label=\"pgp examples in nodejs permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>PGP Examples in Node.js</h2>\n<p>Now, let's go over some examples in Node.js using the <a href=\"https://www.npmjs.com/package/openpgp\">openpgp library</a>.</p>\n<ul>\n<li>OpenPGP is a protocol that defines the standards for PGP. OpenPGP.js implements the OpenPGP protocol in JavaScript.</li>\n</ul>\n<p>We'll go over some basic examples and show how to encrypt &#x26; decrypt large files using Node.js streams.</p>\n<p>First, set up your Node.js project and install openpgp.js:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"0\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">mkdir pgp-tutorial && cd pgp-tutorial && npm init</span>\n<span class=\"grvsc-line\">npm i openpgp --save</span></code></pre>\n<p>Note: examples use openpgp v4.10.8</p>\n<h3 id=\"generating-keys\" style=\"position:relative;\"><a href=\"#generating-keys\" aria-label=\"generating keys permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Generating keys</h3>\n<p>When generating private and public PGP keys with OpenPGP, you can define which curve to use in Elliptic-curve cryptography. In this example, we use Ed25519 for its performance and small key size. For the full list of curves, you can choose from, refer to OpenPGP.js docs.</p>\n<p>You also need to define a passphrase used to decrypt files and the private key. In practice, this should be a strong, randomized secret generated for a single-use.</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"1\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// generate-keys.js</span>\n<span class=\"grvsc-line\">const openpgp = require(&quot;openpgp&quot;);</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">generate();</span>\n<span class=\"grvsc-line\">async function generate() {</span>\n<span class=\"grvsc-line\">  const { privateKeyArmored, publicKeyArmored } = await openpgp.generateKey({</span>\n<span class=\"grvsc-line\">    userIds: [{ name: &quot;person&quot;, email: &quot;person@somebody.com&quot; }],</span>\n<span class=\"grvsc-line\">    curve: &quot;ed25519&quot;,</span>\n<span class=\"grvsc-line\">    passphrase: &quot;qwerty&quot;,</span>\n<span class=\"grvsc-line\">  });</span>\n<span class=\"grvsc-line\">  console.log(privateKeyArmored);</span>\n<span class=\"grvsc-line\">  console.log(publicKeyArmored);</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>Running the above gives us our private key:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"2\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">-----BEGIN PGP PRIVATE KEY BLOCK-----</span>\n<span class=\"grvsc-line\">Version: OpenPGP.js v4.10.8</span>\n<span class=\"grvsc-line\">Comment: https://openpgpjs.org</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">xYYEX6iKVxYJKwYBBAHaRw8BAQdANJ6JIXuMMZV3NIlwq0POS7xsF2N7+kAE</span>\n<span class=\"grvsc-line\">7KQjAtfIuqj+CQMI4CUgW9jPsGPgJvQnnCWFf1s7lO/5+D5ZQ9JK25fUtmQo</span>\n<span class=\"grvsc-line\">WyHX0Ja1ryOoFnvq7u+7fUC0+RAzt8S1xv3eDzazfgNuLtEmufwMyR6wMi78</span>\n<span class=\"grvsc-line\">Kc0ccGVyc29uIDxwZXJzb25Ac29tZWJvZHkuY29tPsKPBBAWCgAgBQJfqIpX</span>\n<span class=\"grvsc-line\">BgsJBwgDAgQVCAoCBBYCAQACGQECGwMCHgEAIQkQVrbGpNEnCPUWIQQb8YRJ</span>\n<span class=\"grvsc-line\">hw7DjekU68lWtsak0ScI9UM7AQDv4YRbIdU2ErPf8MobreeLiXXjYZ6fas8E</span>\n<span class=\"grvsc-line\">zW0KoTZWEQD+NHDY2YYByMF1mWusPkdPDpyBzqMJrlMeihMzZ+PE8AfHiwRf</span>\n<span class=\"grvsc-line\">qIpXEgorBgEEAZdVAQUBAQdARY37/Vys4Sj6DvwN6TRjxrIqiMIngxQgvOb6</span>\n<span class=\"grvsc-line\">wi+tQzEDAQgH/gkDCJ2xNZ1OXxv94E8fTLQ3gYHFQuebn/PSijD8CqlvHNB/</span>\n<span class=\"grvsc-line\">/Z9sIxSFt7rzorW+9v6Awfe+pQwXW5iEyJkdiGu3BM91GMwMvMmZ+rBNlBvq</span>\n<span class=\"grvsc-line\">iX7CeAQYFggACQUCX6iKVwIbDAAhCRBWtsak0ScI9RYhBBvxhEmHDsON6RTr</span>\n<span class=\"grvsc-line\">yVa2xqTRJwj17W0BAI5MuCWHrqjSRcdjLTwxa++jYv+Yxq4tODj8oh27T86v</span>\n<span class=\"grvsc-line\">AQCfb3lij9JGlIMNDQgceeougl+Lw4Gb0kQCnsNQRggTDw==</span>\n<span class=\"grvsc-line\">=yzT4</span>\n<span class=\"grvsc-line\">-----END PGP PRIVATE KEY BLOCK-----</span></code></pre>\n<p>And the public key:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"3\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">-----BEGIN PGP PUBLIC KEY BLOCK-----</span>\n<span class=\"grvsc-line\">Version: OpenPGP.js v4.10.8</span>\n<span class=\"grvsc-line\">Comment: https://openpgpjs.org</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">xjMEX6iKVxYJKwYBBAHaRw8BAQdANJ6JIXuMMZV3NIlwq0POS7xsF2N7+kAE</span>\n<span class=\"grvsc-line\">7KQjAtfIuqjNHHBlcnNvbiA8cGVyc29uQHNvbWVib2R5LmNvbT7CjwQQFgoA</span>\n<span class=\"grvsc-line\">IAUCX6iKVwYLCQcIAwIEFQgKAgQWAgEAAhkBAhsDAh4BACEJEFa2xqTRJwj1</span>\n<span class=\"grvsc-line\">FiEEG/GESYcOw43pFOvJVrbGpNEnCPVDOwEA7+GEWyHVNhKz3/DKG63ni4l1</span>\n<span class=\"grvsc-line\">42Gen2rPBM1tCqE2VhEA/jRw2NmGAcjBdZlrrD5HTw6cgc6jCa5THooTM2fj</span>\n<span class=\"grvsc-line\">xPAHzjgEX6iKVxIKKwYBBAGXVQEFAQEHQEWN+/1crOEo+g78Dek0Y8ayKojC</span>\n<span class=\"grvsc-line\">J4MUILzm+sIvrUMxAwEIB8J4BBgWCAAJBQJfqIpXAhsMACEJEFa2xqTRJwj1</span>\n<span class=\"grvsc-line\">FiEEG/GESYcOw43pFOvJVrbGpNEnCPXtbQEAjky4JYeuqNJFx2MtPDFr76Ni</span>\n<span class=\"grvsc-line\">/5jGri04OPyiHbtPzq8BAJ9veWKP0kaUgw0NCBx56i6CX4vDgZvSRAKew1BG</span>\n<span class=\"grvsc-line\">CBMP</span>\n<span class=\"grvsc-line\">=C6S6</span>\n<span class=\"grvsc-line\">-----END PGP PUBLIC KEY BLOCK-----</span></code></pre>\n<h3 id=\"file-encryption\" style=\"position:relative;\"><a href=\"#file-encryption\" aria-label=\"file encryption permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>File Encryption</h3>\n<p>Now we can start encrypting information.</p>\n<p>Create a text file:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"4\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">echo &#39;This file contains secret information&#39; &gt; secrets.txt</span></code></pre>\n<p>Here, we act as the sender who received a public key from the intended recipient. We use their public key to encrypt the confidential information:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"5\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// encrypt-file.js</span>\n<span class=\"grvsc-line\">const openpgp = require(&quot;openpgp&quot;);</span>\n<span class=\"grvsc-line\">const fs = require(&quot;fs&quot;);</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">const publicKeyArmored = &lt;PUBLIC KEY GIVEN BY RECIPIENT&gt;</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">encrypt();</span>\n<span class=\"grvsc-line\">async function encrypt() {</span>\n<span class=\"grvsc-line\">  const plainData = fs.readFileSync(&quot;secrets.txt&quot;);</span>\n<span class=\"grvsc-line\">  const encrypted = await openpgp.encrypt({</span>\n<span class=\"grvsc-line\">    message: openpgp.message.fromText(plainData),</span>\n<span class=\"grvsc-line\">    publicKeys: (await openpgp.key.readArmored(publicKeyArmored)).keys,</span>\n<span class=\"grvsc-line\">  });</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">  fs.writeFileSync(&quot;encrypted-secrets.txt&quot;, encrypted.data);</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>In the newly created <code>encrypted-secrets.txt</code> file, we have the contents encrypted like so:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"6\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">-----BEGIN PGP MESSAGE-----</span>\n<span class=\"grvsc-line\">Version: OpenPGP.js v4.10.8</span>\n<span class=\"grvsc-line\">Comment: https://openpgpjs.org</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">wV4DUsPKVnc3UHMSAQdAey4TJiEOrZQIrx6q2zBLgmPkbnhPMt1WR+jCWX5x</span>\n<span class=\"grvsc-line\">Gn8wEim8W4OhDVMwfhtgVIClBCGPhvdeZ1zvVUAJGDdl8+S+DUynKhPNcN8m</span>\n<span class=\"grvsc-line\">Kb9TRGYs0sAlAaXcTChBHSS5kDHV/8Hgjcn0OIs6v2mbCkz/bHs/shwf8WMI</span>\n<span class=\"grvsc-line\">ov711iEkgcXnXIX+ZDGyDFnAKftoygzAf0aZy82g7ejAD9SX13wNmO6TK8Gw</span>\n<span class=\"grvsc-line\">wr9Xj8F6XBV0yHvdsm2uzRY9W03tTSqAf0anEs+ZWyVR/ha9ddnZJPFKtUbC</span>\n<span class=\"grvsc-line\">BEF4AMavsIN0CcqpA4q69I3E6GEtkAzgBWfJOOO8mQsNQ1vJWcJocinryBE6</span>\n<span class=\"grvsc-line\">Kbhznoe+R69qmUaJXPpe5scF6tfCYuQtPz4uhOljT+OUP6qss5Nz4zBs4JLq</span>\n<span class=\"grvsc-line\">nUlyynLLSSgdVr4Hvg==</span>\n<span class=\"grvsc-line\">=5tyF</span>\n<span class=\"grvsc-line\">-----END PGP MESSAGE-----</span></code></pre>\n<p>Now, as the sender, we can send the encrypted file to the recipient.</p>\n<h3 id=\"file-decryption\" style=\"position:relative;\"><a href=\"#file-decryption\" aria-label=\"file decryption permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>File Decryption</h3>\n<p>Here, we act as the reciever. To decrypt the <code>encrypted-secrets.txt</code> file, we use our private key and passphrase:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"7\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">// decrypt-file.js</span>\n<span class=\"grvsc-line\">const openpgp = require(&quot;openpgp&quot;);</span>\n<span class=\"grvsc-line\">const fs = require(&quot;fs&quot;);</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">const privateKeyArmored = &lt;PRIVATE KEY&gt;</span>\n<span class=\"grvsc-line\">const passphrase = &lt;PASS PHRASE&gt;;</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">decrypt();</span>\n<span class=\"grvsc-line\">async function decrypt() {</span>\n<span class=\"grvsc-line\">  const privateKey = (await openpgp.key.readArmored([privateKeyArmored])).keys[0];</span>\n<span class=\"grvsc-line\">  await privateKey.decrypt(passphrase);</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">  const encryptedData = fs.readFileSync(&quot;encrypted-secrets.txt&quot;);</span>\n<span class=\"grvsc-line\">  const decrypted = await openpgp.decrypt({</span>\n<span class=\"grvsc-line\">    message: await openpgp.message.readArmored(encryptedData),</span>\n<span class=\"grvsc-line\">    privateKeys: [privateKey],</span>\n<span class=\"grvsc-line\">  });</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">  console.log(decrypted.data);</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>Which logs the decrypted file contents:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"8\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">This file contains secret information.</span></code></pre>\n<h3 id=\"using-streams-for-large-files\" style=\"position:relative;\"><a href=\"#using-streams-for-large-files\" aria-label=\"using streams for large files permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Using Streams for Large Files</h3>\n<p>If you plan on encrypting or decrypting large files, you won't be able to fit the entire file contents in memory. In this case, you can use Node.js streams.</p>\n<p>Here, we encrypt a large file called <code>dataset-1mill.json</code> using streams:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"9\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">encrypt();</span>\n<span class=\"grvsc-line\">async function encrypt() {</span>\n<span class=\"grvsc-line\">  const encrypted = await openpgp.encrypt({</span>\n<span class=\"grvsc-line\">    message: openpgp.message.fromText(fs.createReadStream(&quot;dataset-1mill.json&quot;)),</span>\n<span class=\"grvsc-line\">    publicKeys: (await openpgp.key.readArmored(publicKeyArmored)).keys,</span>\n<span class=\"grvsc-line\">  });</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">  let readStream = encrypted.data;</span>\n<span class=\"grvsc-line\">  let writeStream = fs.createWriteStream(&quot;encrypted-dataset.txt&quot;, { flags: &quot;a&quot; });</span>\n<span class=\"grvsc-line\">  readStream.pipe(writeStream);</span>\n<span class=\"grvsc-line\">  readStream.on(&quot;end&quot;, () =&gt; console.log(&quot;done!&quot;));</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>And then, we decrypt the newly created <code>encrypted-dataset.txt</code> using streams:</p>\n<ul>\n<li>Notice that we set the flag allow<em>unauthenticated</em>stream to true, which allows streaming data before the message integrity has been checked. This is because, in our case, our OpenPGP message only has a single integrity tag at the end. This means the entire message gets loaded into memory, and we get a heap out of memory error since our file is too large to fit into memory at once.</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"\" data-index=\"10\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">openpgp.config.allow_unauthenticated_stream = true;</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">decrypt();</span>\n<span class=\"grvsc-line\">async function decrypt() {</span>\n<span class=\"grvsc-line\">  const privateKey = (await openpgp.key.readArmored([privateKeyArmored])).keys[0];</span>\n<span class=\"grvsc-line\">  await privateKey.decrypt(passphrase);</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">  const decrypted = await openpgp.decrypt({</span>\n<span class=\"grvsc-line\">    message: await openpgp.message.readArmored(fs.createReadStream(&quot;encrypted-dataset.txt&quot;)),</span>\n<span class=\"grvsc-line\">    privateKeys: [privateKey],</span>\n<span class=\"grvsc-line\">  });</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">  let readStream = decrypted.data;</span>\n<span class=\"grvsc-line\">  let writeStream = fs.createWriteStream(&quot;decrypted-dataset.json&quot;, { flags: &quot;a&quot; });</span>\n<span class=\"grvsc-line\">  readStream.pipe(writeStream);</span>\n<span class=\"grvsc-line\">  readStream.on(&quot;end&quot;, () =&gt; console.log(&quot;done!&quot;));</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<p>Now, <code>decrypted-dataset.json</code> will have the same contents as our original <code>dataset-1mill.json</code> file.</p>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n  .dark-default-dark {\n    background-color: #1E1E1E;\n    color: #D4D4D4;\n  }\n</style>","frontmatter":{"title":"Using PGP Encryption with Nodejs","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"November 10, 2020","updated_date":null,"tags":["Security","NodeJs","Encryption"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.7699115044247788,"src":"/static/a427e4132564cc095bcc9015faa67da4/ee604/cover.png","srcSet":"/static/a427e4132564cc095bcc9015faa67da4/69585/cover.png 200w,\n/static/a427e4132564cc095bcc9015faa67da4/497c6/cover.png 400w,\n/static/a427e4132564cc095bcc9015faa67da4/ee604/cover.png 800w,\n/static/a427e4132564cc095bcc9015faa67da4/05d05/cover.png 1080w","sizes":"(max-width: 800px) 100vw, 800px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/using-pgp-encryption-with-nodejs/"}}},{"node":{"id":"3ea7051a-0be9-5995-b1f1-a26c9a436e05","html":"<p><em>This blog is part 1 of a series on gRPC. Part 1 will go over some important concepts, part 2 will be a walkthrough of a client-server implementation in Go, and part 3 will be about LoginRadius' experience migrating to gRPC!</em></p>\n<p><strong>gRPC</strong></p>\n<p>gRPC, simply put, is just another way to send data across networks. It can be used to communicate between services in a microservice architecture, where a single service can interact with multiple others. Similarly, in client-server models, there can be multiple clients communicating with a common backend server.</p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 281px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 108.54092526690391%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAWCAYAAADAQbwGAAAACXBIWXMAAAsTAAALEwEAmpwYAAACnklEQVQ4y6VUW2sTURDOP/KHifgmCAUfvCAqeIGCz4pgKahQJKFSawWt3dLUhrSb0GSzTdo07W6yt+x9z37unM1u0jTRgAPDmT2Z+TJz5pspYEriOOanYRiQJAm9Xg/dbpdrp9NBu93m977vX/HPpDAP0LIsiKKIZrOJer3O7VarhVqthoODg8UBZznNk8xv0r8w7TDpxBi7pnGiACn5cs//z/DbRYxqn3GbxQtmGIYhf6dMXc9HHHromAy3V/t4/73P/SKWZToHkIQAqLuapmEwGMA0TeiGCX+oYVv2cONWBfdf700Axn8HtG2bg2RCtpEAUobnFvDyYxflSit5z/DfTSFxHIfzMCu9UqlAEATOwUzsoQnP8xcDpAyp3NkNS09FUREEcwCn+UT0oLdTVZXr5eUlZFnOvxVF4RVkTZyMLyxCFfqNgBYh+bUMPc+D67pcyc6+aZY5fUb3mQ/dXcswXwimhfPeBXTd4G+o6zrXwUDjC2Hynk4qm+hFwDlgCpYCBk5ClcidWZKhz26SPbR4E69k6AUxxGScypIBoWmhfsEQRCzNPNEgZJDPFP630404V3XYCc3SMRwB7ndCPPni4NWmi3eCh8frDppKlGdBdrHqwPLinDoU/ONYw4O1BpY3TnCiOuOm7J2EKB76WN5ysXUcYF0McNgdAzYSwP3TCJrD8ueRlSHurIh4Wmph6cMxnpcaY8DfpyEelhy8+eXhxVcX9z7baKnRKDhGZxBho+7D9lNANir3WVHC3dUabr6tYuNIHQPSgG9LIZbWbDxKgCtn4UJLdvOwh09CGys/ZTR6Vgo4OzC9OzoSsbOzA2F3F42mhL1yGdVqNRm5YC65C5N7kLG0nKwk2jI0dqRk9/t9zsEoivLVlWkW8wfurJ+AZ2SlUAAAAABJRU5ErkJggg=='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"Screen Shot 2019 10 30 at 1 31 52 PM\"\n        title=\"Screen Shot 2019 10 30 at 1 31 52 PM\"\n        src=\"/static/d43dcaf192be0d2c6332422eaa55f4f2/6b1e2/Screen-Shot-2019-10-30-at-1.31.52-PM.png\"\n        srcset=\"/static/d43dcaf192be0d2c6332422eaa55f4f2/6b1e2/Screen-Shot-2019-10-30-at-1.31.52-PM.png 281w\"\n        sizes=\"(max-width: 281px) 100vw, 281px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<p>The gRPC framework was initially developed at Google and is now open-source. It is a modern implementation of the RPC (Remote Procedure Call) protocol, which has been around since the 80s. You will often see gRPC being compared to other technologies like SOAP, REST, and GraphQL.</p>\n<p>Some features:</p>\n<ul>\n<li>HTTP/2 based transport</li>\n<li>Unary calls: single request, single response</li>\n<li>Streaming calls: client, server, and bidirectional</li>\n<li>Layered design for further extension e.g. authentication, load balancing, logging</li>\n</ul>\n<p><em>How is it used to send data?</em></p>\n<p>gRPC is based on the idea of calling a remote procedure just like a local one. A procedure is like a function or a method. So, ideally developers can treat remote and local calls similarly. A great thing about gRPC is that developers do not need to know the details of the remote interaction.</p>\n<p>Here is a diagram from the official grpc <a href=\"https://grpc.io/docs/guides/\">docs</a> showing the flow:</p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 768px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 62.46153846153847%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAMCAYAAABiDJ37AAAACXBIWXMAAAsTAAALEwEAmpwYAAACh0lEQVQoz3WT+09ScRjG+Uv6rT+g5Q+1ZdkyyQC1ixO1LJzl5hK3Wv1WtvKCm23NVatYmqagycXIazW31BWWFcsmkty8IFJyFBEEuTy950uYkb3bs+85O+d89jzv+x4BtlU8HmdKXvMVDIVgW3DB7HBiyesF5/OBWw/ghrYHBTV1OPe4GRd7X+HCiwFU9r+GgP8oGov9pUg0ymBrgQA+T1sw63bDQ7Cl5WV4SG7fGgrrGpAulWLf5as42q5BZmsnjrd3Q5B0klrrBLPOzyO2w/ONzU3IlC1Ir1FAeF8JiVoHUYcGJzv1CYeDZguevTNCZfwA9fhHNL8dQ/vI2B9AKMzc2l0uzMzNwU4tqOgdQpZKBwlBeGe88gjMgMKbt7E7Nw9pJeexp/gsdokkKGhohG8jxCJOWm0wWb6T4wX4CcyXfOANxAQp0fWS+hgsNwnMv3MXaaVlOHhJjgx5FfbKSnG6VgGT1QorOfKurv4T20V9tHk5OLgV2LlV2Oh0rvgSwDKa0JEWFbLbupgOK9tQpTWw4fBTnXbO4ovFgm/k1P1zGaFwGGHqrT8SRYDeCVBPeW1EIgmgVGNAxlM1xCots53R2oUzKg0WPJ4tRzzEw3Fs2i5SZXMrcq5X41TTAxRT5MJuA2Q9/QngowkTFKNGNBkncG/8E+pH3qNjcgo/CGB2ONgZo3ViYNpLF8XLv1WLQ0VF2H/lGoQdWmRRMjFN+r9rkywvRZ6yO/B1ZobFdi4uwuv3o+K5HgfqG3Hs4ROWTEQDOpFcm2gs/nup42zv+OvYtr8mGdkfDG7dyweHkUnOxNQifmWySTkEFuzkKvUXTE2xSYOoHh6FTN+HcsMgG2r5yyGU0vkLJegoPEJ8Il4AAAAASUVORK5CYII='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"Screen Shot 2019 10 30 at 1 35 45 PM 768x480\"\n        title=\"Screen Shot 2019 10 30 at 1 35 45 PM 768x480\"\n        src=\"/static/a8b70d1d23742fa9aae6c6d2791775c6/e5715/Screen-Shot-2019-10-30-at-1.35.45-PM-768x480.png\"\n        srcset=\"/static/a8b70d1d23742fa9aae6c6d2791775c6/a6d36/Screen-Shot-2019-10-30-at-1.35.45-PM-768x480.png 650w,\n/static/a8b70d1d23742fa9aae6c6d2791775c6/e5715/Screen-Shot-2019-10-30-at-1.35.45-PM-768x480.png 768w\"\n        sizes=\"(max-width: 768px) 100vw, 768px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<p>Each client service will include a stub, which is like an interface containing the available remote procedures. These stubs are auto-generated files.</p>\n<p><em>But what does proto refer to? And what are stubs? How can we generate them?</em></p>\n<p>To answer these questions, we need to look at another technology called protocol buffers.</p>\n<p><strong>Protocol Buffers</strong></p>\n<p>Protocol buffers (protobufs), are a way to format data for storage and transport. gRPC uses protobufs to format data sent over the wire. It is comparable to other data serialization formats such as JSON, XML, and YAML.</p>\n<p>Some features:</p>\n<ul>\n<li>\n<p>Ability to generate interfaces (stubs) for many languages</p>\n<ul>\n<li>You can create proto definitions once (in a .proto file), and compile it into a variety of different languages including Go, Java, C#, and Python.</li>\n</ul>\n</li>\n<li>\n<p>Requires defining schemas - need to know expected data fields and types</p>\n<ul>\n<li>This is unlike JSON which is flexible.</li>\n</ul>\n</li>\n<li>\n<p>Binary format, meaning data is converted into binary when sent over the wire.</p>\n<ul>\n<li>Binary is smaller, and generally can be decoded faster compared to text-based formats such as JSON.</li>\n</ul>\n</li>\n</ul>\n<p><em>How is it used with gRPC?</em></p>\n<p>Step 1: Create proto definitions - methods, requests, responses.</p>\n<ul>\n<li>For example, in account.proto file, we define 3 rpc methods: Find, Update, and Delete.</li>\n<li>These are the remote procedures that can be called by clients.</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"protobuf\" data-index=\"0\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">service AccountService {</span>\n<span class=\"grvsc-line\">  rpc Find(FindRequest) returns (FindResponse);</span>\n<span class=\"grvsc-line\">  rpc Update(UpdateRequest) returns (UpdateResponse);</span>\n<span class=\"grvsc-line\">  rpc Delete(DeleteRequest) returns (DeleteResponse);</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<ul>\n<li>Each method can have its own request and response schemas.</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"protobuf\" data-index=\"1\"><code class=\"grvsc-code\"><span class=\"grvsc-line\">message FindRequest {</span>\n<span class=\"grvsc-line\">  string Uid = 1;</span>\n<span class=\"grvsc-line\">}</span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\">message FindResponse {</span>\n<span class=\"grvsc-line\">  string Uid = 1;</span>\n<span class=\"grvsc-line\">  string Name = 2;</span>\n<span class=\"grvsc-line\">  int32 Age = 3;</span>\n<span class=\"grvsc-line\">  bool isVerified = 4;</span>\n<span class=\"grvsc-line\">}</span></code></pre>\n<ul>\n<li>Note that each field in a message has a unique number. These numbers are used to identify the fields when encoded into binary (recall that protobuf sends data over the wire as binary).</li>\n</ul>\n<p>Step 2: Compile proto file for auto-generated stubs in desired language.</p>\n<ul>\n<li>First, the compiler needs to be <a href=\"https://github.com/protocolbuffers/protobuf\">installed</a>. Refer to blog <em>part 2</em> for detailed instructions.</li>\n<li>The compiler is invoked by the protoc command. In this case, the file account.proto is compiled into Golang with a grpc plugin.</li>\n</ul>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"batch\" data-index=\"2\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk1\">protoc account.proto --go_out=plugins=grpc:.</span></span></code></pre>\n<ul>\n<li>Different languages may have different ways of compiling proto files. For instance with NodeJS, there are npm libraries which allow programmatically compiling proto files.</li>\n</ul>\n<p>Step 3: Use stubs in server and clients.</p>\n<p><strong>The Big Picture</strong></p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 387px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 92.24806201550389%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAASCAYAAABb0P4QAAAACXBIWXMAAAsTAAALEwEAmpwYAAABw0lEQVQ4y3XUhwrjQAwE0Pz/N4YEEkhI7734eIIxG+5OsPZabUcjeQfP57O73W61zudzNx6Pu9lsVt/X67Xe7/e7I76j4zMajbrD4dDHPx6PbuBxv9+77XZbCR1AdzqdylmCJKTf7/fd8XisPf3lcik/S+ygRUim02ktwmYvqf1iseg2m03Z7IfDYfd6verbQX1CAbvdrhSQWkFpL+j7/da3hPwksA8F/CqhQAHKVfpqtSojJ+XEnpLDoxKhDPdii0Mbp0c4JAHBk5MJ3wif0ETkYC+En8+nJx46pHOgF9gmjB769Xpd32LpCiHntsu4VA6+8CQwiCFidyAf9kwD3U+X8UKWy2XxmC4LaudQIHGQCQhdf3UZOiIgQZm7JMzMxaYqCa1MSd9lgZLqHG6UIjE6UrJ99JLN5/OfH+CfXebQdu9/XebDNx3uu+yhQy0XkMYpXU5A9MpXJhH/0+XwkW6nBO8cmCbROdDyp0CaqfhpikCcWG6S/H6TyaRHTQ8VdG4lfPNXhaQ9wlwOmiGYY24Pe4fhMkMvodGC0JglYd9lyRjTwbyhaa+vDHZKjG90hRA/4TFI2wuTreWwrShLBfSQ/gE1JnSTwHNvmgAAAABJRU5ErkJggg=='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"Screen Shot 2019 10 30 at 1 57 55 PM\"\n        title=\"Screen Shot 2019 10 30 at 1 57 55 PM\"\n        src=\"/static/729430686438f756b74979321b48537d/691c3/Screen-Shot-2019-10-30-at-1.57.55-PM.png\"\n        srcset=\"/static/729430686438f756b74979321b48537d/691c3/Screen-Shot-2019-10-30-at-1.57.55-PM.png 387w\"\n        sizes=\"(max-width: 387px) 100vw, 387px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<p><strong>Why gRPC?</strong></p>\n<p>One compelling reason to use gRPC is that it provides high performance</p>\n<ul>\n<li>HTTP/2: e.g. requests are multiplexed, so a single long-lived TCP connection can be used by multiple requests at once. This results in less connection overhead.</li>\n<li>Protobufs: e.g. as a binary format, it allows for quick encoding and decoding of data. </li>\n</ul>\n<p><em>How does it differ from REST?</em></p>\n<table>\n<thead>\n<tr>\n<th></th>\n<th><strong>gRPC</strong></th>\n<th><strong>REST</strong></th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>API</td>\n<td>Contract-based i.e. stubs</td>\n<td>Resource-based and relies on HTTP verbs i.e. GET/PUT/POST/DELETE</td>\n</tr>\n<tr>\n<td>Network protocol</td>\n<td>HTTP/2</td>\n<td>HTTP/1.1 or HTTP/2</td>\n</tr>\n<tr>\n<td>Data serialization format</td>\n<td>Protocol buffers</td>\n<td>JSON</td>\n</tr>\n<tr>\n<td>Streaming</td>\n<td>Built-in support for client, server, and bi-directional streaming</td>\n<td>REST on HTTP/1.1 does not allow streaming</td>\n</tr>\n</tbody>\n</table>\n<p><strong>Other Considerations</strong></p>\n<p>Here, we briefly go over a few other things to consider with gRPC. These will be covered in more detail in another blog.</p>\n<p><em>Management of proto files</em><br>\nIf you plan to have multiple proto files and client services, you need some way to manage them for distribution and version control. One solution is to keep all proto files in a central git repository, and maintain versions using git tags.</p>\n<p><em>Using proto2 vs. proto3</em><br>\nProtocol buffers have two syntax versions: proto2 and proto3.</p>\n<ul>\n<li>One key feature in proto2 is that it differentiates between required and optional fields, and supports nullable fields.</li>\n<li>In proto3, all fields are optional and nullable fields are no longer supported. If a field is unset, it will be set to a default value e.g. empty string for type string. Because of this, determining whether a field was intentionally set or not requires workarounds such as using wrappers.</li>\n</ul>\n<p><em>Load-Balancing</em><br>\nSomething to note about load-balancing gRPC is that HTTP/2 requires L7 (Application Layer) load-balancers. Recall that in HTTP/2, TCP connections are long-lived and requests are multiplexed. This makes L4 (Connection Layer) load-balancers ineffective. This differs from HTTP/1.1 where TCP connections get cycled and can benefit from L4 load-balancers. </p>\n<p>That's it for now! To learn more, check out the official <a href=\"https://grpc.io\">gRPC</a> and <a href=\"https://developers.google.com/protocol-buffers\">protobuf</a> docs.</p>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n  .dark-default-dark {\n    background-color: #1E1E1E;\n    color: #D4D4D4;\n  }\n  .dark-default-dark .mtk1 { color: #D4D4D4; }\n</style>","frontmatter":{"title":"Getting Started with gRPC - Part 1 Concepts","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"October 30, 2019","updated_date":null,"tags":["Engineering","gRPC"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/ceff66d5b341e58e4b85f5a6c3a7cbad/f006c/grpc.png","srcSet":"/static/ceff66d5b341e58e4b85f5a6c3a7cbad/69585/grpc.png 200w,\n/static/ceff66d5b341e58e4b85f5a6c3a7cbad/f006c/grpc.png 246w","sizes":"(max-width: 246px) 100vw, 246px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/getting-started-with-grpc-part-1-concepts/"}}},{"node":{"id":"2644589b-5c0e-5354-a784-84ff04c24fe4","html":"<p><strong>Introduction</strong></p>\n<p>Cross-Site Request Forgery (CSRF) is a common web application attack where a victims’ authenticated session becomes compromised. This attack essentially tricks a victim into performing unintended tasks on a website they are authenticated in. There are variations to this attack, and a popular one we will discuss is utilizing authentication token to imitate api requests.</p>\n<p><strong>Context</strong></p>\n<p>In order to understand CSRF, it is important to know how cookies and authentication tokens are used for persisting user sessions. Cookies are information stored in the browser, and often used for managing state between HTTP requests. A key feature of cookies is that they are automatically passed as a header in HTTP requests. Authentication tokens are typically stored as cookies, and are a way to keep track of a users’ authenticated session. These tokens are set as cookies after a user successfully authenticates themselves by log in.</p>\n<p><strong>How it works</strong></p>\n<p>CSRF takes advantage of the storage of auth tokens in the browser, and constructs http requests to a target server on behalf of the user. Imitating http requests from the legitimate site requires research and preparation from the attacker beforehand, such as finding vulnerable websites and api’s suitable for the attack.</p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 606px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 56.76567656765676%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAYAAAB/Ca1DAAAACXBIWXMAAAsTAAALEwEAmpwYAAABB0lEQVQoz4WTiQrEMAhE8/+/WSj0vu8jyxMmZLvLriBaY8bRWHddlz/P02NjH7nv2yzf67r64zjM7vtuPlaqM8cHINht24IvMAESQ4qisDwVlCLEnRLnefZN05hPNbEUIDFkGAYrGncgC1YAhO44jH8By7I00HgsXwGnafrJUHMCsK7rj7ZDywCRwGy6rgsHxBnDsiy+bVvf973lAUhhzuM5vzHkImAk4gMYJ+MTs7GMo7VMnjYD5ijFHQ5AeZ4bKAy4+HxlPQRMVTxJEp+mqVnFnfYKdBQmaDxDwDmDHW3DUPekoeV4qePlfq4EYFmWGYuqqmyuTzFA/0PiP0XM9VCMisJS5bwAS0depIUcuKgAAAAASUVORK5CYII='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"image2\"\n        title=\"image2\"\n        src=\"/static/b83e1bb3840acfc8b1162e6263c5df22/4d4a2/image2.png\"\n        srcset=\"/static/b83e1bb3840acfc8b1162e6263c5df22/4d4a2/image2.png 606w\"\n        sizes=\"(max-width: 606px) 100vw, 606px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<p>Here is a high-level example of an CSRF attack. Note that some details are excluded for simplicity, but the key aspects are included.</p>\n<ol>\n<li>\n<p>John is authenticated on banking.io</p>\n<ul>\n<li>Auth token is set as a cookie on the browser.</li>\n</ul>\n</li>\n<li>\n<p>On another tab, John clicks on an advertisement for free money, which leads to a malicious site.</p>\n<ul>\n<li>Typically, some social engineering is necessary to lure victims to a malicious website.</li>\n</ul>\n</li>\n<li>\n<p>Malicious site makes a POST request to banking.io/setpassword, which is an api for setting a users password to anything.</p>\n<ul>\n<li>The malicious site will construct the POST request for setting password exactly like the legitimate site, and uses John’s authentication cookie.</li>\n<li>The password will be set to anything the attacker wants.</li>\n</ul>\n</li>\n<li>Victim is unable to authenticate with banking.io anymore, because the password was set to something else.</li>\n</ol>\n<p><strong>Mitigation</strong></p>\n<p>A common and effective way of mitigating CSRF is called the double submit cookie. Essentially the client will have two paired and encrypted tokens: one hidden in the page HTML and the other stored as a cookie.</p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 362px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 69.06077348066299%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsTAAALEwEAmpwYAAABV0lEQVQ4y5WUi66CMAyG9/6PR6LcjKJBoyKCeEHwUvM1GdnhcNSzpGm3tf/6twUjzno+n519uVzkcDhIVVWqr9dr5+P69ZdxwawjIH0AHjifzx9BTT+7pmmkKArJskyF7PI8l/V6ree32+0Xm7eAbdtqIFlaTWbYyP1+/5FlX4y7eTwe6lyWpQp7BJAkSRT80zJDh9Am2DaGeu52O6Ve17UKPmh8uIcFtiFgu93Kfr9XgdZqtdIAd+Ecx7F4niez2UzCMJTRaNTZ4/FY62ygttlspCgLBSSL5XLZAdrakg1NCoJAojiSKIrE931ZLBYymUz0nJhBylA4nU5aP5qEtpTYE0i38bM2D1LrwaaQ+nw+VwBqCTgsCPq6KZYar0DdrSF3ZEZ5/hqXbmyGBpuOTqdTLTjZpmmqjQIQ4H8NNrTdwP63/fWn1weldgAzRujj8djdv/s5vAAlG0QDj7T1eQAAAABJRU5ErkJggg=='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"image1\"\n        title=\"image1\"\n        src=\"/static/d77ad92ba64b2cbbd3e711289b8a6750/10600/image1.png\"\n        srcset=\"/static/d77ad92ba64b2cbbd3e711289b8a6750/10600/image1.png 362w\"\n        sizes=\"(max-width: 362px) 100vw, 362px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<p>When a request is made by the client, both tokens are sent to the server, and the server will then ensure the tokens are valid pairs before processing the request as normal. </p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 360px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 68.61111111111111%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAOCAYAAAAvxDzwAAAACXBIWXMAAAsTAAALEwEAmpwYAAABVElEQVQ4y5WU666CQAyEef9n868aQRMiclMBr9yx5muyBBHMOZs0S0p3djpTsGRivV4v3buuk9vtJpfLRa7Xq+7P5/OrbrisX2BZlknbth/5oijkfr/3uTHoLCBMYOX7voRhKEmS6B4EgTLlwimWs4BlWerBPM+VFUEOduRN3RegoT0OFqxoG7CqqnS3bVsej8dHy8Ow5MfiYNM0CsYzenLJlBk9Q4qjOFJ94jiW8/ksx+NRme12O9VsCJCmqepKLe+oV23DQOWwEJeC1Woly+VSNpuNbLdbBVssFspoqBVGAeI4jra/Xq/1ee/ttZu+ZYAJ2jKHYYNu5Oq61h3AocPkzGj1psw5fTgclD1OM+DoGEWRajp2+MOUKYc5hI6GGXmYMTrk5ybj52CfTidxXVcD3TzPUzMA/Pdgw4oBRujhOy6i/T9/emNQTCDMT8IM9dzP4Q2SS0UPJo5/RQAAAABJRU5ErkJggg=='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"image3\"\n        title=\"image3\"\n        src=\"/static/849c7d576c91bbfd334eee6ffbdf0853/f21e7/image3.png\"\n        srcset=\"/static/849c7d576c91bbfd334eee6ffbdf0853/f21e7/image3.png 360w\"\n        sizes=\"(max-width: 360px) 100vw, 360px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<p>Now the attacker will be unable to perform CSRF since they will not have access to the token hidden in the pages HTML, and the target server requires a valid token pair before processing the request.</p>\n<p>There are also many other mitigation techniques, such as using the Same-Site cookie attribute, and requiring user interaction such as CAPTCHA for requests. Learn more on the <a href=\"https://owasp.org/www-community/attacks/csrf\">OWASP wiki</a>).</p>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n</style>","frontmatter":{"title":"Introduction to Cross-Site Request Forgery (CSRF)","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"October 30, 2019","updated_date":null,"tags":["CSRF"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1.3333333333333333,"src":"/static/4a13eeb54334febf9c8db79b24424eb6/2244e/crosspath.jpg","srcSet":"/static/4a13eeb54334febf9c8db79b24424eb6/f836f/crosspath.jpg 200w,\n/static/4a13eeb54334febf9c8db79b24424eb6/2244e/crosspath.jpg 400w","sizes":"(max-width: 400px) 100vw, 400px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/introduction-to-cross-site-request-forgery-csrf/"}}},{"node":{"id":"4b688594-9414-574b-9a09-fdb408066b30","html":"<p>Both encryption and hashing have significant uses in cryptology and other fields. One defining difference between them is that encryption is reversible, while hashing is irreversible. Because of this, encryption is often used for protecting the confidentiality of data. Only authorized people with the key should be able to access the data. On the other hand, hashing works well for verification; knowing the actual data is unnecessary, just whether or not the hashes are the same.</p>\n<p>Encryption example: sending confidential documents to a co-worker through email.</p>\n<ol>\n<li>Encrypt confidential documents.</li>\n<li>Send encrypted documents &#x26; key to co-worker through different sources.</li>\n<li>Co-worker receives the documents &#x26; decrypts them using the key.</li>\n</ol>\n<p>Hashing example: verifying user credentials for login.</p>\n<ol>\n<li>User registers and creates a password.</li>\n<li>Server hashes a password and stores it in a database.</li>\n<li>User logs in by submitting their password.</li>\n<li>Server hashes the submitted password, and compares it with the hashed password in the database.</li>\n<li>If hashes are the same, the user is authenticated.</li>\n</ol>\n<h2 id=\"encryption\" style=\"position:relative;\"><a href=\"#encryption\" aria-label=\"encryption permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Encryption</h2>\n<p>Encryption is defined as conversion of electronic data into unreadable format by using encryption algorithms. This process of encoding the original data is called encryption. The data dump after encoding is called ciphertext.</p>\n<p>The purpose of encryption is to protect stored data, by guaranteeing that the information cannot be understood by individuals other than the proposed recipient(s).</p>\n<p>Encryption transforms information under another format such that just particular individual(s) could decrypt the conversion.</p>\n<h3 id=\"des\" style=\"position:relative;\"><a href=\"#des\" aria-label=\"des permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>DES</h3>\n<p>The Data Encryption Standard (DES) is a symmetric key algorithm that was widely used for many years. DES is a block cipher that uses a 64-bit block of plaintext and a 56-bit key in order to output a 64-bit block of ciphertext. The core of the algorithm is composed of a series of repetitive modules that transform the block of plaintext. Each module’s bit manipulation includes transposition, splitting, concatenation, and combination with the key. A security limitation is that the key can be brute forced, especially since in DES the key is a relatively short 56-bits (thus, 256possibilities). Because of the technological advances in computing, DES is now considered insecure.</p>\n<h3 id=\"3des\" style=\"position:relative;\"><a href=\"#3des\" aria-label=\"3des permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>3DES</h3>\n<p>Triple Data Encryption Standard (3DES/TDES) is a successor to DES, and runs the DES algorithm three times to each block of data. The standard keying option is to use 3 keys of 56-bits each, resulting in a final key of 3 x 56 = 168-bits. A security limitation is its vulnerability to meet-in-the-middle attacks, where essentially the attacker brute forces the encryption of the plaintext and decryption of the ciphertext at the same time. This allows the 168-bit key to be brute forced in 22 x 56iterations.</p>\n<h3 id=\"aes\" style=\"position:relative;\"><a href=\"#aes\" aria-label=\"aes permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>AES</h3>\n<p>The Advanced Encryption Standard (AES) is a symmetric key algorithm trusted worldwide including the U.S government with classified material. AES is a block cipher which uses 128-bit blocks of plaintext, and three key options: 128-bit, 192-bit, and 256-bit. On a high-level, AES shares many fundamental concepts with DES; in particular, transforming a block of plaintext through repetition and bit manipulation. This include substitution, transposition, and bitwise operations. Currently, the only security limitation is its theoretical risk to brute force.</p>\n<h3 id=\"rsa\" style=\"position:relative;\"><a href=\"#rsa\" aria-label=\"rsa permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>RSA</h3>\n<p>The Rivest-Shamir-Adleman (RSA) is a asymmetric key algorithm based on the difficulty of prime factorization. The algorithm first generates a private and public key using 2 random, sufficiently large, and distinct prime numbers. Public keys can then be distributed to external parties. Plaintext encrypted using the public key and RSA formula can only be decrypted using the private key. Security limitations include weak key generation due to poor choices in prime numbers, and the possibility of breakthroughs such as quantum computers trivializing prime factorization.</p>\n<h3 id=\"blowfish\" style=\"position:relative;\"><a href=\"#blowfish\" aria-label=\"blowfish permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Blowfish</h3>\n<p>Blowfish is a symmetric key algorithm freely available in the public domain. As a block cipher, Blowfish processes 64-bit blocks of plaintext, and a key ranging from 32 to 448-bits. It is known to be fast compared to existing alternatives, except when changing keys. The algorithm involves multiple cycles of splitting the key into 2 subarrays, substituting bits, and performing a series of bitwise operations with parts of the plaintext block. A security limitation is its relatively small block size of 64-bits makes it vulnerable to birthday attacks, which is based on probability theory.</p>\n<h3 id=\"twofish\" style=\"position:relative;\"><a href=\"#twofish\" aria-label=\"twofish permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Twofish</h3>\n<p>Twofish is a symmetric key algorithm freely available in the public domain. Twofish is a block cipher with 128-bit blocks of plaintext, and up to a 256-bit key. The designer of Blowfish also worked on Twofish. Similar to Blowfish, Twofish is a fast cipher, and shares some of the same concepts and structure in transforming a block of plaintext. Currently, the only security limitation is its theoretical risk to brute force.</p>\n<h3 id=\"skipjack\" style=\"position:relative;\"><a href=\"#skipjack\" aria-label=\"skipjack permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Skipjack</h3>\n<p>Skipjack is a symmetric key algorithm with 64-bit blocks of plaintext and 80-bit key. It was designed by the NSA with the purpose of encrypting voice transmission, and later declassified for public knowledge. The algorithm is based off a technique of repeatedly splitting the plaintext block and performing bitwise operations with subkeys. Currently, the only security limitation is its theoretical risk to brute force, especially due to its relatively short key.</p>\n<h3 id=\"use-cases\" style=\"position:relative;\"><a href=\"#use-cases\" aria-label=\"use cases permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Use Cases</h3>\n<p>Symmetric key encryption</p>\n<ul>\n<li>Enforcing confidentiality of data: encryption and decryption of plaintext.</li>\n<li>E.g. protecting top secret documents.</li>\n</ul>\n<p>Asymmetric key encryption</p>\n<ul>\n<li>Key exchange: encrypt a symmetric key which is then used for encrypting and decrypting plaintext.</li>\n<li>Authentication: a single private key and the distribution of multiple public keys.</li>\n<li>Less ideal for encrypting/decrypting plaintext compared to symmetric encryption due to being slower from high overhead.</li>\n<li>E.g. digital signatures; protocols such as SSH, SSL.</li>\n</ul>\n<h2 id=\"hashing\" style=\"position:relative;\"><a href=\"#hashing\" aria-label=\"hashing permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Hashing</h2>\n<p>Hashing is a process of taking a big block of data and reducing it to smaller blocks of data in a specific order by using hashing functions. Cryptographic hashes are irreversible.</p>\n<ul>\n<li>E.g. One way password management, chain management.</li>\n</ul>\n<p>Some properties of hashed data:</p>\n<ul>\n<li>Same inputs will always produce the same outputs.</li>\n<li>Different inputs should not produce the same output (otherwise, a hash collision occurs).</li>\n<li>Input should not be derived from output.</li>\n<li>Small changes to the input should drastically change the output.</li>\n</ul>\n<p>The output of a hashing algorithm is a hashed value, also known as a message digest. Analogous to a fingerprint.</p>\n<h3 id=\"md4\" style=\"position:relative;\"><a href=\"#md4\" aria-label=\"md4 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>MD4</h3>\n<p>The Message Digest 4 (MD4) algorithm takes an input text of arbitrary length, and outputs a 128-bit digest in the form of a 32-digit hexadecimal number. The algorithm works by first padding the text to a certain length, and then appending to it a 64-bit binary representation of the text. Next, the text is processed in blocks of 512-bits, with each block undergoing three rounds of bit manipulation. MD4 is insecure, as a collision attack was found. This is where two input texts produce the same output digest (a hash collision), thus allowing for issues such as forging digital signatures.</p>\n<h3 id=\"md5\" style=\"position:relative;\"><a href=\"#md5\" aria-label=\"md5 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>MD5</h3>\n<p>The Message Digest 5 (MD5) algorithm is similar to MD4, except each block is processed in four more complex rounds. MD5 is also considered insecure, as a collision attack was found. However, MD5 is still often used in the industry for cases which do not require collision resistance, such as password hashing. Better solutions exists, but tradition and lack of modern security expertise drives the popularity of MD5.</p>\n<h3 id=\"sha-1\" style=\"position:relative;\"><a href=\"#sha-1\" aria-label=\"sha 1 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>SHA-1</h3>\n<p>The Secure Hash Algorithm 1 (SHA-1) takes an input text of arbitrary length, and outputs a 160-bit digest, typically in the form of a 40-digit hexadecimal number. The algorithm performs padding, and 80 rounds of text manipulation such as bitwise shifting and XOR operations. SHA-1 is considered insecure, as a collision attack was found.</p>\n<h3 id=\"sha-2\" style=\"position:relative;\"><a href=\"#sha-2\" aria-label=\"sha 2 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>SHA-2</h3>\n<p>The Secure Hash Algorithm 2 (SHA-2) is a family of successors to SHA-1. This includes SHA-224, SHA-256, SHA-384, and SHA-512. Digest sizes range from 224 to 512-bits, increasing its difficulty to brute force. The algorithm consists of padding, and 64 or 80 rounds of bit manipulation. A security limitation is its vulnerability to length extension attacks. When the algorithm is finished, this attack takes advantage of the internal state of the machine in order to keep processing new text. As a result, it is possible to construct a new digest which is an extension of the original digest.</p>\n<h3 id=\"hmac-sha1\" style=\"position:relative;\"><a href=\"#hmac-sha1\" aria-label=\"hmac sha1 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>HMAC-SHA1</h3>\n<p>Hash-based Message Authentication Code SHA-1 (HMAC-SHA1) uses the SHA-1 hashing algorithm and a key in order to generate a HMAC. Due to the usage of a key, there is less chance of a hash collision, but the key is vulnerable to discovery through brute force. Additionally, HMAC is vulnerable to length extension attacks.</p>\n<h3 id=\"hmac-sha256\" style=\"position:relative;\"><a href=\"#hmac-sha256\" aria-label=\"hmac sha256 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>HMAC-SHA256</h3>\n<p>Hash-based Message Authentication Code SHA-256 (HMAC-SHA256) uses the SHA-256 hashing algorithm and a key in order to generate a HMAC. Security concerns include the key being brute forced, and length extension attacks.</p>\n<h3 id=\"pbkdf2\" style=\"position:relative;\"><a href=\"#pbkdf2\" aria-label=\"pbkdf2 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>PBKDF2</h3>\n<p>Password-Based Key Derivation Function 2 (PBKDF2) is a hashing algorithm designed to be used for passwords. By design, hashing using PBKDF2 is slow, making it much more difficult to brute force a password. This is because the algorithm takes in a random salt, as well as the desired number of times to hash the password. Other inputs include the desired length of the output, and the hashing function used. Typically, the recommended number of iterations range in the tens of thousands, but depends on the hashing function and capabilities of the application. However, brute force still remains a threat, especially with weakly chosen salts and a small number of iterations.</p>\n<h3 id=\"argon-2\" style=\"position:relative;\"><a href=\"#argon-2\" aria-label=\"argon 2 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Argon 2</h3>\n<p>Argon2 is a cryptographic hashing algorithm, most recommended for password hashing. It hashes a plain text input to a hash as per the parameters mentioned. It is governed by six parameters: password, salt, memory cost, time cost, parallelism factor, the hash length, along with one of the three algorithms included in it.</p>\n<p>Argon2 has 3 versions: Argon2d, Argon2i and Argon2id.</p>\n<ol>\n<li>Argon2d is more resistant to GPU attacks as it accesses the memory array in a password dependent order reducing the possibility for TMTO attacks but leaves itself vulnerable to side-channel attacks.</li>\n<li>Argon2i, unlike '2d', accesses memory in a password independent order which increases resistance against side-channel attacks.</li>\n<li>Argon2id is a hybrid of '2i' and '2d'. It is always recommended one except when there are reasons to prefer one of the other two modes.</li>\n</ol>\n<p>It has experienced two attacks on Argon2i. The first attack is applicable only to the old version of Argon2i. The second attack has not been secured yet.</p>\n<h3 id=\"use-cases-1\" style=\"position:relative;\"><a href=\"#use-cases-1\" aria-label=\"use cases 1 permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Use Cases</h3>\n<p>Authentication</p>\n<ul>\n<li>Storing and comparing hashed passwords in a database.</li>\n</ul>\n<p>Message integrity</p>\n<ul>\n<li>A person sends a message, as well as its corresponding hash (likely through a different source). The receiver can hash the message, and if the hashes are different then the message was compromised.</li>\n</ul>\n<p>Identification</p>\n<ul>\n<li>Database indexing.</li>\n</ul>\n<h1 id=\"encoding-and-cryptography\" style=\"position:relative;\"><a href=\"#encoding-and-cryptography\" aria-label=\"encoding and cryptography permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Encoding and Cryptography</h1>\n<h3 id=\"encoding\" style=\"position:relative;\"><a href=\"#encoding\" aria-label=\"encoding permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Encoding</h3>\n<p>The process of transforming the data by using an algorithm (that is publicly available) into another format. </p>\n<p>The motivation behind encoding is to change information with the goal that it can be appropriately (and securely) fed to a different system. The main objective is not to keep data secret, but instead to guarantee that it is ready to be legitimately used.</p>\n<h3 id=\"symmetric-key-cryptography\" style=\"position:relative;\"><a href=\"#symmetric-key-cryptography\" aria-label=\"symmetric key cryptography permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Symmetric key cryptography</h3>\n<p>The process of using the same key for encrypting and decrypting the text is called symmetric key cryptography.</p>\n<h3 id=\"asymmetric-key-cryptography\" style=\"position:relative;\"><a href=\"#asymmetric-key-cryptography\" aria-label=\"asymmetric key cryptography permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Asymmetric key cryptography</h3>\n<p>The process of using a public key for encryption and a private key for decryption is called asymmetric key cryptography.</p>\n<h3 id=\"stream-cipher\" style=\"position:relative;\"><a href=\"#stream-cipher\" aria-label=\"stream cipher permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Stream cipher</h3>\n<p>The process of encrypting or decrypting the text bit by bit using a symmetric key is called stream cipher. The stream cipher process is high speed and requires low hardware complexity.</p>\n<h3 id=\"block-cipher\" style=\"position:relative;\"><a href=\"#block-cipher\" aria-label=\"block cipher permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Block cipher</h3>\n<p>The process of encrypting or decrypting the text block by block using a symmetric key is called block cipher. Block ciphers are the functions that take an input message and a key in order to create a new, encrypted ciphertext. Block cipher are used with Symmetric key encryption.</p>\n<p>Block ciphers are invertible and efficiently computable. E.g. DES, AES, BlowFish etc.</p>\n<h3 id=\"cryptographic-salt\" style=\"position:relative;\"><a href=\"#cryptographic-salt\" aria-label=\"cryptographic salt permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a>Cryptographic Salt</h3>\n<p>Salts are an additional piece of data used in hashing algorithms, typically for passwords. They help protect against brute force attacks, by adding complexity to the hashes. As a result, salts increase the time taken to brute force a single hash, and deter against optimizations such as dictionaries and precomputed tables.</p>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n</style>","frontmatter":{"title":"Encryption and Hashing","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"December 24, 2018","updated_date":null,"tags":["Encryption","Hashing"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":3.278688524590164,"src":"/static/9f60d8a12e2cb9240bfe54b54515b42a/3087f/encryption-and-hashing.png","srcSet":"/static/9f60d8a12e2cb9240bfe54b54515b42a/69585/encryption-and-hashing.png 200w,\n/static/9f60d8a12e2cb9240bfe54b54515b42a/497c6/encryption-and-hashing.png 400w,\n/static/9f60d8a12e2cb9240bfe54b54515b42a/3087f/encryption-and-hashing.png 615w","sizes":"(max-width: 615px) 100vw, 615px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/encryption-and-hashing/"}}},{"node":{"id":"ced9f3b5-ac7d-5aea-a05f-96f4f5eeff80","html":"<p>Time Required: 20 minutes.<br>\nTechnologies: Express, Node.js, JavaScript.<br>\nPrerequisites:</p>\n<ul>\n<li>Basic knowledge of Express, Node.js, and JavaScript.</li>\n<li>Node.js and npm are installed.</li>\n</ul>\n<p>This tutorial will go over how to build a bot that will respond to pings (i.e. @<bot-name>), and send messages to a chat room. On a high level, the bot will run on an express server, and receive pings via an HTTP endpoint. Responses to pings will be sent synchronously through a payload in the HTTP response, while bot-initiated messages will be sent asynchronously using the Google Hangout Chat API.</p>\n<h5 id=\"outline\" style=\"position:relative;\"><a href=\"#outline\" aria-label=\"outline permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a><strong>Outline</strong></h5>\n<ol>\n<li>Environment setup.</li>\n<li>Get bot to respond to pings.</li>\n<li>Send bot-initiated messages.</li>\n<li>Deploy.</li>\n</ol>\n<h5 id=\"environment-setup\" style=\"position:relative;\"><a href=\"#environment-setup\" aria-label=\"environment setup permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a><strong>Environment Setup</strong></h5>\n<p>Create a new project with the file ‘app.js’.<br>\nOpen command line/terminal, and navigate to your project directory. Run ‘npm init’, and press enter until package.json is created. Next, install the following dependencies:</p>\n<ul>\n<li>express: <code>npm install express --save</code></li>\n<li>body-parser:  <code>npm install body-parser --save</code></li>\n<li>googleapis: <code>npm install googleapis --save</code></li>\n<li>unirest:  <code>npm install unirest --save</code></li>\n</ul>\n<p>In ‘app.js’, let’s setup our server:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"js\" data-index=\"0\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk4\">const</span><span class=\"mtk1\"> </span><span class=\"mtk12\">express</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">require</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;express&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">const</span><span class=\"mtk1\"> </span><span class=\"mtk12\">bodyParser</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">require</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;body-parser&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">const</span><span class=\"mtk1\"> { </span><span class=\"mtk12\">google</span><span class=\"mtk1\"> } = </span><span class=\"mtk11\">require</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;googleapis&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk4\">const</span><span class=\"mtk1\"> </span><span class=\"mtk12\">app</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">express</span><span class=\"mtk1\">();</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk12\">app</span><span class=\"mtk1\">.</span><span class=\"mtk11\">use</span><span class=\"mtk1\">(</span><span class=\"mtk12\">bodyParser</span><span class=\"mtk1\">.</span><span class=\"mtk11\">urlencoded</span><span class=\"mtk1\">({</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk12\">extended:</span><span class=\"mtk1\"> </span><span class=\"mtk4\">false</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}));</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk12\">app</span><span class=\"mtk1\">.</span><span class=\"mtk11\">use</span><span class=\"mtk1\">(</span><span class=\"mtk12\">bodyParser</span><span class=\"mtk1\">.</span><span class=\"mtk11\">json</span><span class=\"mtk1\">());</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk12\">app</span><span class=\"mtk1\">.</span><span class=\"mtk11\">listen</span><span class=\"mtk1\">(</span><span class=\"mtk7\">8100</span><span class=\"mtk1\">, </span><span class=\"mtk4\">function</span><span class=\"mtk1\">() {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk10\">console</span><span class=\"mtk1\">.</span><span class=\"mtk11\">log</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;App listening on port 8100.&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">});</span></span></code></pre>\n<p>Running ‘node app.js’ will now create a local server on port 8100.</p>\n<h5 id=\"responding-to-pings\" style=\"position:relative;\"><a href=\"#responding-to-pings\" aria-label=\"responding to pings permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a><strong>Responding to Pings</strong></h5>\n<p>The bot will respond to pings through a HTTP POST endpoint. Create one with express:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"js\" data-index=\"1\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk12\">app</span><span class=\"mtk1\">.</span><span class=\"mtk11\">post</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;/&#39;</span><span class=\"mtk1\">, </span><span class=\"mtk4\">function</span><span class=\"mtk1\">(</span><span class=\"mtk12\">req</span><span class=\"mtk1\">, </span><span class=\"mtk12\">res</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk10\">console</span><span class=\"mtk1\">.</span><span class=\"mtk11\">log</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;someone pinged @&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk15\">if</span><span class=\"mtk1\"> (</span><span class=\"mtk12\">req</span><span class=\"mtk1\">.</span><span class=\"mtk12\">body</span><span class=\"mtk1\">.</span><span class=\"mtk12\">type</span><span class=\"mtk1\"> === </span><span class=\"mtk8\">&#39;MESSAGE&#39;</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">    </span><span class=\"mtk15\">return</span><span class=\"mtk1\"> </span><span class=\"mtk12\">res</span><span class=\"mtk1\">.</span><span class=\"mtk11\">json</span><span class=\"mtk1\">({</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      </span><span class=\"mtk12\">text:</span><span class=\"mtk1\"> </span><span class=\"mtk8\">&#39;sleeping...&#39;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">    });</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  }</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">});</span></span></code></pre>\n<p>The bot will respond with the text: ‘sleeping…’.</p>\n<p>Synchronously responding to messages simply requires us to return a response to Google. The downside to this is the 30 second time limit before Google no longer accepts responses to the request. For instance, this would be a problem if you were building some kind of reminder app; the bot wouldn't be able to synchronously respond after 30 seconds. This is where async responses come in.</p>\n<h5 id=\"bot-initiated-messages\" style=\"position:relative;\"><a href=\"#bot-initiated-messages\" aria-label=\"bot initiated messages permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a><strong>Bot-initiated Messages</strong></h5>\n<p>To show this, we will have our bot post to a chat room every 1 minute.</p>\n<p>Sending async messages to Google API requires a Service Account for authentication. Once authenticated, we can make a POST request to a Google API URL that will create a message.</p>\n<p>So first, create a Google Service Account following these <a href=\"https://developers.google.com/hangouts/chat/how-tos/service-accounts\">steps</a>. Take the downloaded JSON file and put it in the root directory of your project. Here, we renamed it to googlekeys.json:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"js\" data-index=\"2\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk4\">const</span><span class=\"mtk1\"> </span><span class=\"mtk12\">gkeys</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">require</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;./googlekeys.json&#39;</span><span class=\"mtk1\">);</span></span></code></pre>\n<p>We will be making POST requests using Unirest:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"js\" data-index=\"3\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk4\">const</span><span class=\"mtk1\"> </span><span class=\"mtk12\">unirest</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">require</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;unirest&#39;</span><span class=\"mtk1\">);</span></span></code></pre>\n<p>Now generate a JWT that will be used in our POST request:</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"js\" data-index=\"4\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk4\">function</span><span class=\"mtk1\"> </span><span class=\"mtk11\">getJWT</span><span class=\"mtk1\">() {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk15\">return</span><span class=\"mtk1\"> </span><span class=\"mtk4\">new</span><span class=\"mtk1\"> </span><span class=\"mtk10\">Promise</span><span class=\"mtk1\">(</span><span class=\"mtk4\">function</span><span class=\"mtk1\">(</span><span class=\"mtk12\">resolve</span><span class=\"mtk1\">, </span><span class=\"mtk12\">reject</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">    </span><span class=\"mtk4\">let</span><span class=\"mtk1\"> </span><span class=\"mtk12\">jwtClient</span><span class=\"mtk1\"> = </span><span class=\"mtk4\">new</span><span class=\"mtk1\"> </span><span class=\"mtk10\">google</span><span class=\"mtk1\">.</span><span class=\"mtk10\">auth</span><span class=\"mtk1\">.</span><span class=\"mtk10\">JWT</span><span class=\"mtk1\">(</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      </span><span class=\"mtk12\">gkeys</span><span class=\"mtk1\">.</span><span class=\"mtk12\">client_email</span><span class=\"mtk1\">,</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      </span><span class=\"mtk4\">null</span><span class=\"mtk1\">,</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      </span><span class=\"mtk12\">gkeys</span><span class=\"mtk1\">.</span><span class=\"mtk12\">private_key</span><span class=\"mtk1\">, [</span><span class=\"mtk8\">&#39;https://www.googleapis.com/auth/chat.bot&#39;</span><span class=\"mtk1\">]</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">    );</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">    </span><span class=\"mtk12\">jwtClient</span><span class=\"mtk1\">.</span><span class=\"mtk11\">authorize</span><span class=\"mtk1\">(</span><span class=\"mtk4\">function</span><span class=\"mtk1\">(</span><span class=\"mtk12\">err</span><span class=\"mtk1\">, </span><span class=\"mtk12\">tokens</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      </span><span class=\"mtk15\">if</span><span class=\"mtk1\"> (</span><span class=\"mtk12\">err</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">        </span><span class=\"mtk10\">console</span><span class=\"mtk1\">.</span><span class=\"mtk11\">log</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;Error create JWT hangoutchat&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">        </span><span class=\"mtk11\">reject</span><span class=\"mtk1\">(</span><span class=\"mtk12\">err</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      } </span><span class=\"mtk15\">else</span><span class=\"mtk1\"> {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">        </span><span class=\"mtk11\">resolve</span><span class=\"mtk1\">(</span><span class=\"mtk12\">tokens</span><span class=\"mtk1\">.</span><span class=\"mtk12\">access_token</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      }</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">    });</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  });</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}</span></span></code></pre>\n<p>Here is our function for posting messages. ROOM-ID can be found in the URL of the hangout chat room page i.e.: <code>https://chat.google.com/u/0/room/{ROOM-ID}</code></p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"js\" data-index=\"5\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk4\">function</span><span class=\"mtk1\"> </span><span class=\"mtk11\">postMessage</span><span class=\"mtk1\">(</span><span class=\"mtk12\">count</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk15\">return</span><span class=\"mtk1\"> </span><span class=\"mtk4\">new</span><span class=\"mtk1\"> </span><span class=\"mtk10\">Promise</span><span class=\"mtk1\">(</span><span class=\"mtk4\">function</span><span class=\"mtk1\">(</span><span class=\"mtk12\">resolve</span><span class=\"mtk1\">, </span><span class=\"mtk12\">reject</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      </span><span class=\"mtk11\">getJWT</span><span class=\"mtk1\">().</span><span class=\"mtk11\">then</span><span class=\"mtk1\">(</span><span class=\"mtk4\">function</span><span class=\"mtk1\">(</span><span class=\"mtk12\">token</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">          </span><span class=\"mtk12\">unirest</span><span class=\"mtk1\">.</span><span class=\"mtk11\">post</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;https://chat.googleapis.com/v1/spaces/&#39;</span><span class=\"mtk1\"> + {ROOM-</span><span class=\"mtk12\">ID</span><span class=\"mtk1\">} + </span><span class=\"mtk8\">&#39;/messages&#39;</span><span class=\"mtk1\">)</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">              .</span><span class=\"mtk11\">headers</span><span class=\"mtk1\">({</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">                  </span><span class=\"mtk8\">&quot;Content-Type&quot;</span><span class=\"mtk12\">:</span><span class=\"mtk1\"> </span><span class=\"mtk8\">&quot;application/json&quot;</span><span class=\"mtk1\">,</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">                  </span><span class=\"mtk8\">&quot;Authorization&quot;</span><span class=\"mtk12\">:</span><span class=\"mtk1\"> </span><span class=\"mtk8\">&quot;Bearer &quot;</span><span class=\"mtk1\"> + </span><span class=\"mtk12\">token</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">              })</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">              .</span><span class=\"mtk11\">send</span><span class=\"mtk1\">(</span><span class=\"mtk10\">JSON</span><span class=\"mtk1\">.</span><span class=\"mtk11\">stringify</span><span class=\"mtk1\">({</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">                  </span><span class=\"mtk8\">&#39;text&#39;</span><span class=\"mtk12\">:</span><span class=\"mtk1\"> </span><span class=\"mtk8\">&#39;Hello! This is message number &#39;</span><span class=\"mtk1\"> + </span><span class=\"mtk12\">count</span><span class=\"mtk1\">,</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">              }))</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">              .</span><span class=\"mtk11\">end</span><span class=\"mtk1\">(</span><span class=\"mtk4\">function</span><span class=\"mtk1\">(</span><span class=\"mtk12\">res</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">                  </span><span class=\"mtk11\">resolve</span><span class=\"mtk1\">();</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">              });</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      }).</span><span class=\"mtk11\">catch</span><span class=\"mtk1\">(</span><span class=\"mtk4\">function</span><span class=\"mtk1\">(</span><span class=\"mtk12\">err</span><span class=\"mtk1\">) {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">          </span><span class=\"mtk11\">reject</span><span class=\"mtk1\">(</span><span class=\"mtk12\">err</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      });</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  });</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">}</span></span></code></pre>\n<p>Finally, add the code that will repeat our post every minute.</p>\n<pre class=\"grvsc-container dark-default-dark\" data-language=\"js\" data-index=\"6\"><code class=\"grvsc-code\"><span class=\"grvsc-line\"><span class=\"mtk4\">const</span><span class=\"mtk1\"> </span><span class=\"mtk12\">timer</span><span class=\"mtk1\"> = </span><span class=\"mtk11\">require</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;timers&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk12\">app</span><span class=\"mtk1\">.</span><span class=\"mtk11\">listen</span><span class=\"mtk1\">(</span><span class=\"mtk7\">8100</span><span class=\"mtk1\">, </span><span class=\"mtk4\">function</span><span class=\"mtk1\">() {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk10\">console</span><span class=\"mtk1\">.</span><span class=\"mtk11\">log</span><span class=\"mtk1\">(</span><span class=\"mtk8\">&#39;App listening on port 8100.&#39;</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk4\">let</span><span class=\"mtk1\"> </span><span class=\"mtk12\">count</span><span class=\"mtk1\"> = </span><span class=\"mtk7\">0</span><span class=\"mtk1\">;</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  </span><span class=\"mtk12\">timer</span><span class=\"mtk1\">.</span><span class=\"mtk11\">setInterval</span><span class=\"mtk1\">(</span><span class=\"mtk4\">function</span><span class=\"mtk1\">() {</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">      </span><span class=\"mtk11\">postMessage</span><span class=\"mtk1\">(</span><span class=\"mtk12\">count</span><span class=\"mtk1\"> += </span><span class=\"mtk7\">1</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">  }, </span><span class=\"mtk7\">60000</span><span class=\"mtk1\">);</span></span>\n<span class=\"grvsc-line\"><span class=\"mtk1\">});</span></span></code></pre>\n<h5 id=\"deploy\" style=\"position:relative;\"><a href=\"#deploy\" aria-label=\"deploy permalink\" class=\"anchor before\"><svg aria-hidden=\"true\" focusable=\"false\" height=\"16\" version=\"1.1\" viewBox=\"0 0 16 16\" width=\"16\"><path fill-rule=\"evenodd\" d=\"M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z\"></path></svg></a><strong>Deploy</strong></h5>\n<p>Expose your local server to public (we used <a href=\"https://ngrok.com/\">ngrok</a>).</p>\n<p>Login to <a href=\"https://console.developers.google.com\">developer console</a>. Create a new project, and enable Hangout Chat API. Under configuration, set:</p>\n<ul>\n<li>status: live</li>\n<li>bot name (this is how you will add and ping the bot)</li>\n<li>avatar</li>\n<li>description</li>\n<li>functionality: rooms</li>\n<li>connection settings - bot URL: <your ngrok HTTPS url></li>\n<li>permission: everyone in your domain</li>\n</ul>\n<p>Restart your local server, and that’s it! Make sure you have your bot added to the chat room, and you can ping it by sending @<bot-name>. The bot will also post to the chat room every minute.</p>\n<p>There are a lot of different ways to further extend this bot, such as setting reminders/notifications, making to-do lists, displaying server logs, and interacting with API’s.</p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 456px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 98.90350877192984%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAIAAAAC64paAAAACXBIWXMAAAsTAAALEwEAmpwYAAABiElEQVQ4y41TybKCMBDk/7/M7faKix60WAKIEBKW7Pga8/QdlDJdldSkoGcmneloGAattZRSCPEMpPTnb4jwm1IKtHmePd85Z7Hu3xFh7ff77XZrrT2dToSQprmlOakon6aJc450q2R8Y4wVZQky6zpGadu212tVkLyu6zzPca9VMlr1kXFOODtaI99anlfwTxbGZD2Pm2s+8HsYIkpp9wBnrGd84Bw7jiwAi9poYFHYWr8/A+Nh17GofT6fsyxDiqIoUBPpWD9OLuyp4jg+HA6Q/Xy55IQUhCRAtqidpinSec0+kNHYn57YnZvRs7UYGyEmP3D+h8+VX9+kNeU4HGlTjn2o2n4wAWVMK8Z67JkUXkKPeR0RZhD8ZbyVgsBo22qjHnj4Q76Cd0T+PZYK91kbDYv4WvdAY2w2m91uhxTH47GqKkwLKatbvzjM+2yVjJodpTAA+l080TR4IXgrTRMkCnoqAN2qhzfM7ELVfpEnoxPOfuoy4V0o2d8KOhmtobbDMBuEQfgFyzeOgOEE8E8AAAAASUVORK5CYII='); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"image2\"\n        title=\"image2\"\n        src=\"/static/01e633c666a77beb405600b9e10f32a8/7f664/image2.png\"\n        srcset=\"/static/01e633c666a77beb405600b9e10f32a8/7f664/image2.png 456w\"\n        sizes=\"(max-width: 456px) 100vw, 456px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<p><span\n      class=\"gatsby-resp-image-wrapper\"\n      style=\"position: relative; display: block; margin-left: auto; margin-right: auto; max-width: 300px; \"\n    >\n      <span\n    class=\"gatsby-resp-image-background-image\"\n    style=\"padding-bottom: 41.333333333333336%; position: relative; bottom: 0; left: 0; background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAICAIAAAB2/0i6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAAwElEQVQY042POQ7CMBBFfX6OQAEn4Aq0IBaJggYIFFaUhGyO4yz2jD0mRNQkrxzp//mP9X3vvSdH1lpnf6CdBQMDy/NiF25FXgevJ+dcCFG22vlpmDGwua0vye7dmSCNizwf7+TnhBEh4gloVHWtlALdhVW3uumotd+OvyVsmL4/7JMkTrPsfn9UomzAHVNQQJMDmNbaEqJDtGiMQcQxQkRuAnKDs7mmp7jhUsimbfx84+GzlDIqwrzKBuV61J7PB66zz8uuwzWBAAAAAElFTkSuQmCC'); background-size: cover; display: block;\"\n  ></span>\n  <img\n        class=\"gatsby-resp-image-image\"\n        alt=\"image1\"\n        title=\"image1\"\n        src=\"/static/0f2b5c8c3421fb1370ecf03c036718fe/5a46d/image1.png\"\n        srcset=\"/static/0f2b5c8c3421fb1370ecf03c036718fe/5a46d/image1.png 300w\"\n        sizes=\"(max-width: 300px) 100vw, 300px\"\n        style=\"width:100%;height:100%;margin:0;vertical-align:middle;position:absolute;top:0;left:0;\"\n        loading=\"lazy\"\n      />\n    </span></p>\n<style class=\"grvsc-styles\">\n  .grvsc-container {\n    overflow: auto;\n    -webkit-overflow-scrolling: touch;\n    padding-top: 1rem;\n    padding-top: var(--grvsc-padding-top, var(--grvsc-padding-v, 1rem));\n    padding-bottom: 1rem;\n    padding-bottom: var(--grvsc-padding-bottom, var(--grvsc-padding-v, 1rem));\n    border-radius: 8px;\n    border-radius: var(--grvsc-border-radius, 8px);\n    font-feature-settings: normal;\n  }\n  \n  .grvsc-code {\n    display: inline-block;\n    min-width: 100%;\n  }\n  \n  .grvsc-line {\n    display: inline-block;\n    box-sizing: border-box;\n    width: 100%;\n    padding-left: 1.5rem;\n    padding-left: var(--grvsc-padding-left, var(--grvsc-padding-h, 1.5rem));\n    padding-right: 1.5rem;\n    padding-right: var(--grvsc-padding-right, var(--grvsc-padding-h, 1.5rem));\n  }\n  \n  .grvsc-line-highlighted {\n    background-color: var(--grvsc-line-highlighted-background-color, transparent);\n    box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, transparent);\n  }\n  \n  .dark-default-dark {\n    background-color: #1E1E1E;\n    color: #D4D4D4;\n  }\n  .dark-default-dark .mtk4 { color: #569CD6; }\n  .dark-default-dark .mtk1 { color: #D4D4D4; }\n  .dark-default-dark .mtk12 { color: #9CDCFE; }\n  .dark-default-dark .mtk11 { color: #DCDCAA; }\n  .dark-default-dark .mtk8 { color: #CE9178; }\n  .dark-default-dark .mtk7 { color: #B5CEA8; }\n  .dark-default-dark .mtk10 { color: #4EC9B0; }\n  .dark-default-dark .mtk15 { color: #C586C0; }\n</style>","frontmatter":{"title":"Creating a Google Hangout Bot with Express and Node.js","author":{"id":"Andy Yeung","github":null,"avatar":null},"date":"June 06, 2018","updated_date":null,"tags":["Express","NodeJs","Hangout"],"coverImage":{"childImageSharp":{"fluid":{"aspectRatio":1,"src":"/static/f62c49828f8a2a1d1483abd689ceb93e/aac8d/hangoutchatimgage2.jpg","srcSet":"/static/f62c49828f8a2a1d1483abd689ceb93e/f836f/hangoutchatimgage2.jpg 200w,\n/static/f62c49828f8a2a1d1483abd689ceb93e/2244e/hangoutchatimgage2.jpg 400w,\n/static/f62c49828f8a2a1d1483abd689ceb93e/aac8d/hangoutchatimgage2.jpg 630w","sizes":"(max-width: 630px) 100vw, 630px"}}}},"fields":{"authorId":"Andy Yeung","slug":"/engineering/creating-a-google-hangout-bot-with-express-and-node-js/"}}}]},"authorYaml":{"id":"Andy Yeung","bio":"Software Developer at LoginRadius with an interest in big data and basketball..","github":null,"stackoverflow":null,"linkedin":null,"medium":null,"twitter":null,"avatar":null}},"pageContext":{"id":"Andy Yeung","__params":{"id":"andy-yeung"}}},"staticQueryHashes":["1171199041","1384082988","2100481360","23180105","528864852"]}