Thomas R. Koll

Bitbucket pipeline with an elasticsearch server

2023-09-12T00:00:00+02:00

This week I’m setting up a bitbucket pipeline for a Ruby on Rails app that uses a few services, among them elastic search. Now starting that service throws up an error as elasticsearch servers are social animals and really like to connect to other elasticsearch servers and you’ll see this erro in the log/tab for the elasticsearch server:

the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes] must be configured

The error doesn’t provide the answer or rather configuration you need as it is discovery.type=single-node what you want to use. I also use that option in the docker-compose.yml for local development on that app.

Simple you think, just add this to the bitbucket-pipelines.yml? Not quite because if you do this is the error you get from bitbucket pipelines and on top of that, bitbucket’s own validator is just fine with it. Yeah, I guess there’s two different pieces of code for the yml file…

There is an error in your bitbucket-pipelines.yml at [definitions > services > elasticsearch]. The variable name “discovery.type” is invalid. It should contain only alphanumeric characters and underscores and it should not begin with a number.

After quite some research a stackoverflow answer pointed me into the right direction, elasticsearch has become more flexible about their configuration, starting with version 7.15 (I have to run 7.17, lucky me) and you can substitute discovery.type with ES_SETTING_DISCOVERY_TYPE giving you the following bitbucket-pipelines.yml:

definitions:
  services:
    elasticsearch:
      image: elasticsearch:7.17.6
      variables:
        ES_SETTING_DISCOVERY_TYPE: single-node

HDD Data Rescue - Fixing a HDD with a burnt cable

2022-10-03T00:00:00+02:00

Last month my girlfriend’s Seagate Barracuda 7200.12 500GB HDD went up in smoke, luckily it was just the drive’s power cable in that dusty PC, but the damage was done and I wasn’t keen on just replacing the cable. After reading a bit into the topic I figured I could do the data rescue myself as a burnt cable and PCB usually just involved replacing the PCB to regain access to all the data. So Igot a replacement disk of the same model and same revision number of the PCB, this is important, removed the PCB of both discs, located the ROM chip, de-soldered it on both and put the chip from the fried PCB onto the new PCB that I installed on the old drive and it did work!

The ROM had a 25FU406B 07 1PHOS written on it. Note that my PCB had another 8-legged IC on it but with a very different number on it. ROMs are easy to spot once you have a general idea about their number patterns.

A SMD soldering station is highly recommended, it’s just so much easier than with a plain soldering iron.

You’ll see on the photos that the ROM isn’t that perfectly on its pads, but I used a magnifying glass to visually confirm there were no shorts and it did work. Sloppy work is just alright, eh?

The disk was then copied onto a fresh disk using ddrescue, part of the SystemRescue Linux distribution and almost no bad sectors were discovered.

Lastly: This whole operation was just to get the data from the HDD. The disk will not be used beyond that and be replaced with a new HDD!

Here’s a few photos with details of the damage, chips etc.

The smoke makes it look worse than it is, except for the plastic connector the PCB might be fine and re-usable (I will not))

Looks like the cable powering the HDD spontaneously developed a short?

It was very dusty after all

The burnt PCB

The HDDs in comparison, look at all that dust!

In case of this PCB the ROM was located at the corner

The revisions are etched into the PCB. Note that the numbers on the HDD sticker might be the same but the PCB revisions still be different like REV A vs REV B

The meaty side of the PCBs in comparison. Different chips but all we care about is the revision

<img src=’/assets/hdd-data-rescue/thumbnails/hdd-pcb-without-the-rom.jpg’ alt=’Here’s the ROM chip de-soldered ‘ /> Here’s the ROM chip de-soldered

And the ROM soldered onto the replacement PCB. Not the most perfect job but it will do for the rescue operation

The results of ddrescue, barely anything was lost

Sign-in options - good practices

2022-09-13T12:01:43+02:00

A bit of a rant to be honest, but as I’m currently once again moving from one android phone to the other, there’s a few things in terms of UX that I’d like to complain about from worst to best:

No data transfer at all

This is pretty much the default with most apps and if you have more than one instagram account like I do, you’ll hate signing into all of those accounts. I understand this behaviour for something like banking apps, but insta or any other non-sensitive app?

I didn’t see this with any app, but how about just storing the user name in the cloud and when opening the app on the new device as for the passwort? Certainly an interesting option for sensitive apps

Copy data via export/import

K9 Mail is an example for this, still a bit annoying as you have to copy the file from one phone to the other, but certainly makes having multiple mail accounts with complicated setups more easy.

QR code to copy data

Now this is of course only an option if you still have the old device. I’ve seen a few apps that put my credentials and data into a QR code on the old device that I had to read with the new device and was all set up. Google Authenticator is the best example for this and it’s just brilliant. The learning app my kid uses for schools also has this feature, who would have thought?

Sync the data via the cloud

A few non-sensitive apps had this and I’m happy to have this feature with those apps but wouldn’t want it with my banking app. BBC Sounds, Strava or any other low-key app would benefit from this sort of UX behaviour.

Conclusio

The levels of user convenience vary a lot and tend to the bad end of none at all. But as we know, implementing a simple QR code with either the necessary data or just a reference to data on a server is very simple. I did this for Budgetfuchs a few years ago to allow users to sign-in from a mobile device more quickly and to invite others to use the app with them.

Cleaning bad laptop speakers

2019-05-20T08:01:43+02:00

Over the last months the speakers of my Thinkpadx X240 got worse and worse. Sure, half-broken bottom case with screws missing didn’t help but the sound was just awful when turned up to 11.

So when I finally replaced that bottom case (€30 well spent) I finally had a look into the speaker issue and had the idea it might be just dirty. Luckily I tried to clean it with my somewhat magnetic screwdriver and small bits of metal unstuck from the speaker and stuck to the screwdriver. Yay. A few minutes the two speakers were all nice and clean again and the sound just perfect once more.

Below are pictures from before and after.

Rails url helpers as liquid filters

2019-05-14T00:00:00+02:00

For a new personal project, Checklistenguru.de I wanted to use liquid for the longish description texts that get filled with links to other pages inside the app. Naturally the usual url helpers get used and I had to come up with a way to use them.

I got the inspiration from Philippe Bourgau and the following lines will allow you to use any named route _path in a liquid filter, e.g. {{"moon-trip" | public_template_list_path }}. If you also set the default_url_options you will be able to use the _url helpers as well.

# app/helpers/link_helper.rb
# yes, not in the initializers but the helpers

module LinkFilter
  include Rails.application.routes.url_helpers
  def default_url_options
    {}
  end
end

Liquid::Template.register_filter(LinkFilter)

Pundit with inherited resources

2019-04-22T00:00:00+02:00

Call be old-fashioned and out-dated, but I’m still a huge fan of inherited_resources and of ourse I’m using it in my latest project which will be about checklists.

I also added pundit and pundit_extra to the project and when using the latter you’ll run into some conflict with inherited_resources method resource. The load_resource from pundit_extra ultimately uses instance_variable_get "@#{resource_name}" to get the resource and so I replace the method with my own.

  def resource_instance
    if params[:id]
      resource
    else
      build_resource
    end
  end

ej2 DataManager with Rails

2019-03-16T19:01:59+01:00

Recently I used syncfusion’s ej2 calendar in a small Rails project and ran into a few problems because Rails is tough on security.

The UrlAdaptor that comes with ej2-data needs to send our beloved csrf-token with all POST requests and this is the way I implemented it:

    class UrlAdaptorWithCredentials extends UrlAdaptor {
      beforeSend(args: DataManager, xhr: XMLHttpRequest) {
        xhr.withCredentials = true;
        var token = document.querySelector('meta[name="csrf-token"]').content
        xhr.setRequestHeader('X-CSRF-Token', token)
        super.beforeSend(args, xhr);
      }
    }
    data = new DataManager({
      url:       '/calendar',
      crossDomain: true,
      adaptor: new UrlAdaptorWithCredentials()
    });

Lastly a general advice on fetch: With rails you’ll need to send the cookie for GET requests and you can do this by adding {credentials: "include"} in the options.

    fetch('/calendar', {credentials: "include"})

Rails credentials gone bad

2019-03-05T00:00:00+01:00

It’d be boring if you didn’t start you day with a nice error message, no matter what I tried, even editing the credentials gave an error. Here’s the one I got today, and some quick look into ActiveSupport::MessageEncryptor showed me that something was wrong with the credentials.

rescue in _decrypt': ActiveSupport::MessageEncryptor::InvalidMessage

The credentials.yml.enc had been modified in the repo just slightly during a merge conflict and though they were now the same again, it didn’t play along. Maybe a newline or something?

Anyways, I went straight for a solution by resetting the repo to before the change to the credentials.yml.enc file, run rails credentials:edit to copy its contents into a temporary file, back to the terminal, reset to HEAD run rails credentials:edit again and paste in the contents from the temporary file.

Easy, right?

What to do when your Linux upgrade goes wrong

2019-01-23T00:00:00+01:00

Strange how rarely I write about Linux, the OS that I’m using since 1998 (with some gap filled with OSX). I love it to bits because it’s is super reliable both on the desktop and server. I even have one machine with SteamOS installed. But of course there are exceptions like 20 years ago when I was using Debian testing and libpam had a bad update. Took me a week to get back into my system. The olden days without backup-systems…

This week’s outtake was much shorter and completely my fault, but my backup-strategy was a second PC. Not the perfect solution if the situations change and I have no access to another machine or the internet.

First of all, what did I do wrong? Well, I downgraded readline, don’t ask, I just had to. Bash didn’t like that and insisted on readline 8. A fact I realized only after booting the system when I came home from work.

So, what took you so long? Well, not being able to get into my system to reinstall the newer readline version that was still in pacman’s cache. My disks aren’t ecrypted (files are, there’s even a mountable encrypted disk image) so recovery was easy enough.

What will I change in the near future? First off all, a newer, bigger disk is on my shopping-list. I’l configure it to feature an additional, minimalistic boot drive that I can use to access my other partitions when needed. Another fallback will be changing the external hard-drives that I have around the house, I’ll add boot drives to those as well. You might wonder why that, but the reason is quite simple: I don’t want to rely on an USB stick for a boot drive because I probably will overwrite it after a while.

Stuck shutter problem: Olympus PEN E-PL5

2019-01-05T00:00:00+01:00

There’s an anoying mechanical problem with a few cameras like my Olympus PEN E-PL5 which results in a shutter stuck when trying to take a photo. Someone even made a video, and others report the same issue too. The camera’s film function works as expected.

The camera has to shutter curtains according to this comment on mikrocontroller.net and while the first one works without a hitch, the other one is the one prune to get stuck.

My solution

So, once you are sure the problem description fits and you can safely assume it’s a mechanical problem, then I have a solution for you. It requires applying some force. You get yourself a decent sized book and place it on the table, the book is what you’ll slam the camera onto.

Best to do this without a lense installed.

First start the camera, set a long exposure time, click the button and then slam the camera on the book. You might have to repeat this a few times or slam it on a different side.

I had the issue twice and that’s how I solved it both times.

If it worked for you, please do send me a mail and let me know.

Update 2019-03-26: It happened again, but a 5s exposure and and whack on the knee loosened the shutter.