pim/blog
2
0
Fork 1
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Improve about page

Browse source 
Remove cache and compiled site
This commit is contained in:
Pim Kunis 2024-04-30 13:12:09 +02:00
parent 77d432ad8b
commit 03cf79e8f0
82 changed files with 321 additions and 1405 deletions
Download patch file Download diff file Expand all files Collapse all files

180
.jekyll-cache/Jekyll/Cache/Jekyll--Converters--Markdown/0e/763d8239a78fde13ad37342cacbcd9ada77932aa71a300ff4a742b3a5403a2 Normal file
View file

@ -0,0 +1,180 @@
I"±N<p>Recently, I deployed <a href="https://concourse-ci.org/">Concourse CI</a> because I wanted to get my feet wet with a CI/CD pipeline.
However, I had a practical use case lying around for a long time: automatically compiling my static website and deploying it to my docker Swarm.
This took some time getting right, but the result works like a charm (<a href="https://git.kun.is/pim/static">source code</a>).</p>
<p>It’s comforting to know I don’t have move a finger and my website is automatically deployed.
However, I would still like to receive some indication of what’s happening.
And what’s a better way to do that, than using my <a href="https://github.com/caronc/apprise">Apprise</a> service to keep me up to date.
There’s a little snag though: I could not find any Concourse resource that does this.
That’s when I decided to just create it myself.</p>
<h1 id="the-plagiarism-hunt">The Plagiarism Hunt</h1>
<p>As any good computer person, I am lazy.
I’d rather just copy someone’s work, so that’s what I did.
I found <a href="https://github.com/mockersf/concourse-slack-notifier">this</a> GitHub repository that does the same thing but for Slack notifications.
For some reason it’s archived, but it seemed like it should work.
I actually noticed lots of repositories for Concourse resource types are archived, so not sure what’s going on there.</p>
<h1 id="getting-to-know-concourse">Getting to know Concourse</h1>
<p>Let’s first understand what we need to do reach our end goal of sending Apprise notifications from Concourse.</p>
<p>A Concourse pipeline takes some inputs, performs some operations on them which result in some outputs.
These inputs and outputs are called <em>resources</em> in Concourse.
For example, a Git repository could be a resource.
Each resource is an instance of a <em>resource type</em>.
A resource type therefore is simply a blueprint that can create multiple resources.
To continue the example, a resource type could be “Git repositoryâ€<C3A2>.</p>
<p>We therefore need to create our own resource type that can send Apprise notifications.
A resource type is simply a container that includes three scripts:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">check</code>: check for a new version of a resource</li>
<li><code class="language-plaintext highlighter-rouge">in</code>: retrieve a version of the resource</li>
<li><code class="language-plaintext highlighter-rouge">out</code>: create a version of the resource</li>
</ul>
<p>As Apprise notifications are basically fire-and-forget, we will only implement the <code class="language-plaintext highlighter-rouge">out</code> script.</p>
<h1 id="writing-the-out-script">Writing the <code class="language-plaintext highlighter-rouge">out</code> script</h1>
<p>The whole script can be found <a href="https://git.kun.is/pim/concourse-apprise-notifier/src/branch/master/out">here</a>, but I will explain the most important bits of it.
Note that I only use Apprise’s persistent storage solution, and not its stateless solution.</p>
<p>Concourse provides us with the working directory, which we <code class="language-plaintext highlighter-rouge">cd</code> to:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cd</span> <span class="s2">"</span><span class="k">${</span><span class="nv">1</span><span class="k">}</span><span class="s2">"</span>
</code></pre></div></div>
<p>We create a timestamp, formatted in JSON, which we will use for the resource’s new version later.
Concourse requires us to set a version for the resource, but since Apprise notifications don’t have that, we use the timestamp:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">timestamp</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-n</span> <span class="s2">"{version:{timestamp:</span><span class="se">\"</span><span class="si">$(</span><span class="nb">date</span> +%s<span class="si">)</span><span class="se">\"</span><span class="s2">}}"</span><span class="si">)</span><span class="s2">"</span>
</code></pre></div></div>
<p>First some black magic Bash to redirect file descriptors.
Not sure why this is needed, but I copied it anyways.
After that, we create a temporary file holding resource’s parameters.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">exec </span>3&gt;&amp;1
<span class="nb">exec </span>1&gt;&amp;2
<span class="nv">payload</span><span class="o">=</span><span class="si">$(</span><span class="nb">mktemp</span> /tmp/resource-in.XXXXXX<span class="si">)</span>
<span class="nb">cat</span> <span class="o">&gt;</span> <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span> &lt;&amp;0
</code></pre></div></div>
<p>We then extract the individual parameters.
The <code class="language-plaintext highlighter-rouge">source</code> key contains values how the resource type was specified, while the <code class="language-plaintext highlighter-rouge">params</code> key specifies parameters for this specific resource.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">apprise_host</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-r</span> <span class="s1">'.source.host'</span> &lt; <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">apprise_key</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-r</span> <span class="s1">'.source.key'</span> &lt; <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">alert_body</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-r</span> <span class="s1">'.params.body'</span> &lt; <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">alert_title</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-r</span> <span class="s1">'.params.title // null'</span> &lt; <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">alert_type</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-r</span> <span class="s1">'.params.type // null'</span> &lt; <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">alert_tag</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-r</span> <span class="s1">'.params.tag // null'</span> &lt; <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span>
<span class="nv">alert_format</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span>jq <span class="nt">-r</span> <span class="s1">'.params.format // null'</span> &lt; <span class="s2">"</span><span class="k">${</span><span class="nv">payload</span><span class="k">}</span><span class="s2">"</span><span class="si">)</span><span class="s2">"</span>
</code></pre></div></div>
<p>We then format the different parameters using JSON:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">alert_body</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">eval</span> <span class="s2">"printf </span><span class="se">\"</span><span class="k">${</span><span class="nv">alert_body</span><span class="k">}</span><span class="se">\"</span><span class="s2">"</span> | jq <span class="nt">-R</span> <span class="nt">-s</span> .<span class="si">)</span><span class="s2">"</span>
<span class="o">[</span> <span class="s2">"</span><span class="k">${</span><span class="nv">alert_title</span><span class="k">}</span><span class="s2">"</span> <span class="o">!=</span> <span class="s2">"null"</span> <span class="o">]</span> <span class="o">&amp;&amp;</span> <span class="nv">alert_title</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">eval</span> <span class="s2">"printf </span><span class="se">\"</span><span class="k">${</span><span class="nv">alert_title</span><span class="k">}</span><span class="se">\"</span><span class="s2">"</span> | jq <span class="nt">-R</span> <span class="nt">-s</span> .<span class="si">)</span><span class="s2">"</span>
<span class="o">[</span> <span class="s2">"</span><span class="k">${</span><span class="nv">alert_type</span><span class="k">}</span><span class="s2">"</span> <span class="o">!=</span> <span class="s2">"null"</span> <span class="o">]</span> <span class="o">&amp;&amp;</span> <span class="nv">alert_type</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">eval</span> <span class="s2">"printf </span><span class="se">\"</span><span class="k">${</span><span class="nv">alert_type</span><span class="k">}</span><span class="se">\"</span><span class="s2">"</span> | jq <span class="nt">-R</span> <span class="nt">-s</span> .<span class="si">)</span><span class="s2">"</span>
<span class="o">[</span> <span class="s2">"</span><span class="k">${</span><span class="nv">alert_tag</span><span class="k">}</span><span class="s2">"</span> <span class="o">!=</span> <span class="s2">"null"</span> <span class="o">]</span> <span class="o">&amp;&amp;</span> <span class="nv">alert_tag</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">eval</span> <span class="s2">"printf </span><span class="se">\"</span><span class="k">${</span><span class="nv">alert_tag</span><span class="k">}</span><span class="se">\"</span><span class="s2">"</span> | jq <span class="nt">-R</span> <span class="nt">-s</span> .<span class="si">)</span><span class="s2">"</span>
<span class="o">[</span> <span class="s2">"</span><span class="k">${</span><span class="nv">alert_format</span><span class="k">}</span><span class="s2">"</span> <span class="o">!=</span> <span class="s2">"null"</span> <span class="o">]</span> <span class="o">&amp;&amp;</span> <span class="nv">alert_format</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">eval</span> <span class="s2">"printf </span><span class="se">\"</span><span class="k">${</span><span class="nv">alert_format</span><span class="k">}</span><span class="se">\"</span><span class="s2">"</span> | jq <span class="nt">-R</span> <span class="nt">-s</span> .<span class="si">)</span><span class="s2">"</span>
</code></pre></div></div>
<p>Next, from the individual parameters we construct the final JSON message body we send to the Apprise endpoint.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">body</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">cat</span> <span class="o">&lt;&lt;</span><span class="no">EOF</span><span class="sh">
{
"body": </span><span class="k">${</span><span class="nv">alert_body</span><span class="k">}</span><span class="sh">,
"title": </span><span class="k">${</span><span class="nv">alert_title</span><span class="k">}</span><span class="sh">,
"type": </span><span class="k">${</span><span class="nv">alert_type</span><span class="k">}</span><span class="sh">,
"tag": </span><span class="k">${</span><span class="nv">alert_tag</span><span class="k">}</span><span class="sh">,
"format": </span><span class="k">${</span><span class="nv">alert_format</span><span class="k">}</span><span class="sh">
}
</span><span class="no">EOF
</span><span class="si">)</span><span class="s2">"</span>
</code></pre></div></div>
<p>Before sending it just yet, we compact the JSON and remove any values that are <code class="language-plaintext highlighter-rouge">null</code>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">compact_body</span><span class="o">=</span><span class="s2">"</span><span class="si">$(</span><span class="nb">echo</span> <span class="s2">"</span><span class="k">${</span><span class="nv">body</span><span class="k">}</span><span class="s2">"</span> | jq <span class="nt">-c</span> <span class="s1">'.'</span><span class="si">)</span><span class="s2">"</span>
<span class="nb">echo</span> <span class="s2">"</span><span class="nv">$compact_body</span><span class="s2">"</span> | jq <span class="s1">'del(..|nulls)'</span> <span class="o">&gt;</span> /tmp/compact_body.json
</code></pre></div></div>
<p>Here is the most important line, where we send the payload to the Apprise endpoint.
It’s quite straight-forward.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl <span class="nt">-v</span> <span class="nt">-X</span> POST <span class="nt">-T</span> /tmp/compact_body.json <span class="nt">-H</span> <span class="s2">"Content-Type: application/json"</span> <span class="s2">"</span><span class="k">${</span><span class="nv">apprise_host</span><span class="k">}</span><span class="s2">/notify/</span><span class="k">${</span><span class="nv">apprise_key</span><span class="k">}</span><span class="s2">"</span>
</code></pre></div></div>
<p>Finally, we print the timestamp (fake version) in order to appease the Concourse gods.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo</span> <span class="s2">"</span><span class="k">${</span><span class="nv">timestamp</span><span class="k">}</span><span class="s2">"</span> <span class="o">&gt;</span>&amp;3
</code></pre></div></div>
<h1 id="building-the-container">Building the Container</h1>
<p>As said earlier, to actually use this script, we need to add it to a image.
I won’t be explaining this whole process, but the source can be found <a href="https://git.kun.is/pim/concourse-apprise-notifier/src/branch/master/pipeline.yml">here</a>.
The most important take-aways are these:</p>
<ul>
<li>Use <code class="language-plaintext highlighter-rouge">concourse/oci-build-task</code> to build a image from a Dockerfile.</li>
<li>Use <code class="language-plaintext highlighter-rouge">registry-image</code> to push the image to an image registry.</li>
</ul>
<h1 id="using-the-resource-type">Using the Resource Type</h1>
<p>Using our newly created resource type is surprisingly simple.
I use it for the blog you are reading right now and the pipeline definition can be found <a href="https://git.kun.is/pim/static/src/branch/main/pipeline.yml">here</a>.
Here we specify the resource type in a Concourse pipeline:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">resource_types</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">apprise</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">registry-image</span>
<span class="na">source</span><span class="pi">:</span>
<span class="na">repository</span><span class="pi">:</span> <span class="s">git.kun.is/pim/concourse-apprise-notifier</span>
<span class="na">tag</span><span class="pi">:</span> <span class="s2">"</span><span class="s">1.1.1"</span>
</code></pre></div></div>
<p>We simply have to tell Concourse where to find the image, and which tag we want.
Next, we instantiate the resource type to create a resource:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="na">resources</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">apprise-notification</span>
<span class="na">type</span><span class="pi">:</span> <span class="s">apprise</span>
<span class="na">source</span><span class="pi">:</span>
<span class="na">host</span><span class="pi">:</span> <span class="s">https://apprise.kun.is:444</span>
<span class="na">key</span><span class="pi">:</span> <span class="s">concourse</span>
<span class="na">icon</span><span class="pi">:</span> <span class="s">bell</span>
</code></pre></div></div>
<p>We simply specify the host to send Apprise notifications to.
Yeah, I even gave it a little bell because it’s cute.</p>
<p>All that’s left to do, is actually send the notification.
Let’s see how that is done:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="na">name</span><span class="pi">:</span> <span class="s">deploy-static-website</span>
<span class="na">plan</span><span class="pi">:</span>
<span class="pi">-</span> <span class="na">task</span><span class="pi">:</span> <span class="s">deploy-site</span>
<span class="na">config</span><span class="pi">:</span> <span class="s">...</span>
<span class="na">on_success</span><span class="pi">:</span>
<span class="err"> </span><span class="na">put</span><span class="pi">:</span> <span class="s">apprise-notification</span>
<span class="na"> params</span><span class="pi">:</span>
<span class="err"> </span> <span class="na">title</span><span class="pi">:</span> <span class="s2">"</span><span class="s">Static</span><span class="nv"> </span><span class="s">website</span><span class="nv"> </span><span class="s">deployed!"</span>
<span class="err"> </span> <span class="na">body</span><span class="pi">:</span> <span class="s2">"</span><span class="s">New</span><span class="nv"> </span><span class="s">version:</span><span class="nv"> </span><span class="s">$(cat</span><span class="nv"> </span><span class="s">version/version)"</span>
<span class="err"> </span><span class="na">no_get</span><span class="pi">:</span> <span class="no">true</span>
</code></pre></div></div>
<p>As can be seen, the Apprise notification can be triggered when a task is executed successfully.
We do this using the <code class="language-plaintext highlighter-rouge">put</code> command, which execute the <code class="language-plaintext highlighter-rouge">out</code> script underwater.
We set the notification’s title and body, and send it!
The result is seen below in my Ntfy app, which Apprise forwards the message to:
<img src="ntfy.png" alt="picture showing my Ntfy app with the Apprise notification" /></p>
<p>And to finish this off, here is what it looks like in the Concourse web UI:
<img src="pipeline.png" alt="the concourse web gui showing the pipeline of my static website including the the apprise notification resources" /></p>
<h1 id="conclusion">Conclusion</h1>
<p>Concourse’s way of representing everything as an image/container is really interesting in my opinion.
A resource type is quite easily implemented as well, although Bash might not be the optimal way to do this.
I’ve seen some people implement it in Rust, which might be a good excuse to finally learn that language :)</p>
<p>Apart from Apprise notifications, I’m planning on creating a resource type to deploy to a Docker swarm eventually.
This seems like a lot harder than simply sending notifications though.</p>
:ET