user – Boris blog! http://idesmanb.com Just another boring site Sun, 16 Oct 2022 14:38:56 +0000 en-US hourly 1 https://wordpress.org/?v=6.3.2 http://idesmanb.com/wp-content/uploads/2020/12/cropped-logo-32x32.bmp user – Boris blog! http://idesmanb.com 32 32 DBT 1.0.0 on AWS MWAA http://idesmanb.com/2022/10/16/dbt-1-0-0-on-aws-mwaa/?utm_source=rss&utm_medium=rss&utm_campaign=dbt-1-0-0-on-aws-mwaa http://idesmanb.com/2022/10/16/dbt-1-0-0-on-aws-mwaa/#comments Sun, 16 Oct 2022 14:25:01 +0000 http://idesmanb.com/?p=461 Read More »]]> Initially I thought that it sounds too good to be true: https://docs.aws.amazon.com/mwaa/latest/userguide/samples-dbt.html

Then I decided to give it a try – only to realise that dbt-core=0.21.1 ?!

Then I went for dbt-core=1.3.0 – it didn’t work -> collision with apache-ariflow=2.2.2

Then I tried dbt-core=1.0.0 – that one worked. It will work for you too, just use the following requirements.txt

apache-airflow[postgres,mysql,google]==2.2.2

json-rpc==1.13.0
minimal-snowplow-tracker==0.0.2
packaging==20.9
networkx==2.6.3 
mashumaro==2.9
sqlparse==0.4.2

logbook==1.5.3
agate==1.6.1
dbt-extractor==0.4.0

pyparsing==2.4.7 
msgpack==1.0.4
parsedatetime==2.6
pytimeparse==1.1.8
leather==0.3.4
pyyaml==6.0

# Airflow constraints are jsonschema==3.2.0
jsonschema==3.1.1
hologram==0.0.14
dbt-core==1.0.0

psycopg2-binary==2.8.6
dbt-postgres==1.0.0
dbt-redshift==1.0.0
]]>
http://idesmanb.com/2022/10/16/dbt-1-0-0-on-aws-mwaa/feed/ 1
Device Setup Failed’ or ‘Unknown Device. Fix for HP 103 107a 108 on Windows http://idesmanb.com/2021/07/12/windows-cannot-find-your-hp-printer-anymore-fix-for-103-107a-108/?utm_source=rss&utm_medium=rss&utm_campaign=windows-cannot-find-your-hp-printer-anymore-fix-for-103-107a-108 http://idesmanb.com/2021/07/12/windows-cannot-find-your-hp-printer-anymore-fix-for-103-107a-108/#comments Mon, 12 Jul 2021 11:05:31 +0000 http://idesmanb.com/?p=442 Read More »]]> Out of a sudden, without any notice, my HP 107a stop communicating with my laptop. Ok, my guess is that some of the latest windows updates crashed it, however no evidence here. As usuall printers stop work just when you need them the most, and I spent good amount of time digging blog posts on how to fix the issue by myself – starting from suggestions for reistalling of the drivers, trough cleaning of the register, hard reset, changing cabels, testing on other machines… (by the way, apparently HP 107a does NOT work with MAC and it is almost impossible to get it to work with Ubuntu, but there is a post here: https://askubuntu.com/questions/1279136/using-hp-laser-107a-with-ubuntu-20-04 )

Long story short, here is the solution that worked for me:

  1. Make sure your printer is connected with the laptop.
  2. Press the Windows key and search for printers and scanners.
  3. Select the item and click on Add a printer or scanner. Your laptop will start searchng, but after a few seconds you will see an option: The printer that I want isn’t listed.
  4. Go for it. Screenshots follow 🙂

In the dialog pick Add a local printer or network printer with manual settings.

Go to the HP section and select your driver (you should already have it, remember, your printer worked 😉 If not, you can istall it from https://support.hp.com/us-en/drivers/selfservice/hp-laser-100-printer-series/24494339/model/24494340

Then, pick Use the dirver that is currently installed (recommended).

Give a name.

And that’s all! You should be able to print a test page and proceed with using the printer just as before.

If nothing else works, call the HP support. I did it and they helped me. Kudos to the guy who walked this setup with me.

]]>
http://idesmanb.com/2021/07/12/windows-cannot-find-your-hp-printer-anymore-fix-for-103-107a-108/feed/ 2
Atlantis for Terraform with support for automated building of python AWS Lambda Functions [Dockerfile] http://idesmanb.com/2021/02/17/atlantis-terraform-python-lambda-aws/?utm_source=rss&utm_medium=rss&utm_campaign=atlantis-terraform-python-lambda-aws http://idesmanb.com/2021/02/17/atlantis-terraform-python-lambda-aws/#comments Wed, 17 Feb 2021 07:42:23 +0000 http://idesmanb.com/?p=417 Read More »]]> If you look for a way to use Atlantis both to deploy your infrastructure as a code and package and build AWS Lambda Functions with Python I provide you a Dockerfile for that purpose. Some use cases require, that the Terraform repository holds the infra as code and also the actual source code that this infra should execute. In the case of the Lambda functions, that is both the description for a function (how much memory, timeout, IAM role) as well as the actual code that has to be executed. On the other hand python lambda functions has to be built and zipped locally, especially if they contain external libraries, before uploading them to the cloud. That means, the machine, or cluster that runs Atlantis needs Python to perform the task, and that is what the following image gives you: it pulls the original Atlantis image from Docker Hub and on top of that installs Python.

Important: You probably want to include this (or similar) terraform module to build the python packages: https://github.com/claranet/terraform-aws-lambda

Dockerfile:

FROM runatlantis/atlantis:latest
ENV USER atlantis_deploy
RUN adduser -D $USER
ENV HOME /home/$USER
ENV PY_VERSION='3.7.2'
RUN apk add --no-cache --update \
    curl git bash bash-doc bash-completion \
    openssh libffi-dev \
    openssl-dev \
    bzip2-dev zlib-dev zip \
    readline-dev sqlite-dev \
    build-base \
    linux-headers \
    util-linux \
    docker
WORKDIR /home/$USER
# Install Python 3.7 via pyenv
ARG PYENV_HOME=$HOME/.pyenv
RUN git clone --depth 1 https://github.com/pyenv/pyenv.git $PYENV_HOME && \
    rm -rfv $PYENV_HOME/.git
ENV PATH $PYENV_HOME/shims:$PYENV_HOME/bin:$PATH
RUN pyenv install $PY_VERSION
RUN pyenv global $PY_VERSION
RUN pip install --upgrade pip && pyenv rehash
RUN rm -rf ~/.cache/pip
# Install additional software
RUN pip3 install --upgrade awscli pipenv boto3 click ssm-cache

]]>
http://idesmanb.com/2021/02/17/atlantis-terraform-python-lambda-aws/feed/ 1
Bad Bayersoien and around: Hinteres Hörnle, Kolbensattelhütte, Schleierfälle http://idesmanb.com/2021/02/10/bad-bayersoien-and-around-hinteres-hornle-kolbensattelhutte-schleierfalle/?utm_source=rss&utm_medium=rss&utm_campaign=bad-bayersoien-and-around-hinteres-hornle-kolbensattelhutte-schleierfalle Wed, 10 Feb 2021 19:46:18 +0000 http://idesmanb.com/?p=299 Read More »]]> In the late spring, there is nothing better than go hiking in the nature. Bad Bayersoien is deffinitely a nice place that I recommend as a starting point, especially for trips in the Ammergau Alps, known as one of the most beautiful monutains in Germany.

From Munich, it will take you around one and a half hours by car to reach Bad Bayersoien. You can also take the train BRB to Füssen and from there a bus (33 stops). In the area, you will find many gast hauses, hotels, restaurants and plenty of nice places to chill or take your kids too. There is even a small lake, Bayersoiener See, perfect for short walks.

Hinteres Hörnle

On the way to Hinteres Hörnle

Hinteres Hörnle (1548m) means Rear Hörnle and on the way to it you can visit two more peaks: Mittleres and Vorderes Hörnle (middle and front Hörnle) as well as the Hörnlehütte (Hörnle hut). There are many nice views on the way to Rear Hörnle as well as forests, small rivers and grasslands. You can visit all three peaks in one day as this is a rather easy hike.

Kolbensattelhütte

Kolbensattelhütte

The Kolbensattelhütte (1270m) is a prefered winter destination because there are some ski tracks, but it is very nice when I paid it a visit, in the late spring as well. You can do a picnic just on the grasslands or have a bier in the hut. You can reach it from Bad Bayersoien in just around an hour and a half and continue hiking in the forest or to the near peeks Am Zahn (1615m), Sonnenberg(1622m), Steckenberg (1385m)

Schleierfälle

The Schleierfälle waterfalls

The Schleierfälle waterfalls are an incredible destination for the hot days of the early summer, because in order to reach them, one would have a nice journey trough the forests. Be careful, just a 100 meter before reaching them the road becomes very slippery because of the small streams that cross it. And then, when you have enought steering at the waterfall you can have a beer on the riverside. Enjoy it!

]]>
Flops bay http://idesmanb.com/2021/01/15/flops-bay-post/?utm_source=rss&utm_medium=rss&utm_campaign=flops-bay-post http://idesmanb.com/2021/01/15/flops-bay-post/#comments Fri, 15 Jan 2021 20:49:10 +0000 http://idesmanb.com/?p=166 Read More »]]> “Failure is simply the opportunity to begin again, this time more intelligently.” – Henry Ford

Hellooo to everyone who asks himself how to buy a failed invention and why somebody would be even selling one?! Soon in this blog there will be a section that will allow you to do exactly that: find something that you never thought it even exists, buy it, and on top of that: meet great people, who think out of the box so much, that no one has been able to find a purpose for their creations… yet.

In short, I want to provide a platform where people can share their failed projects. Ideas do not just disappear, usually what is left can be a business plan, a software product, even some gadget or prototype that is just too cool to be thrown away. And what is even more exciting is, sometimes failed creations can fly again and even receive a fresh boost!

So let’s see..

Update: check out the bay here: http://idesmanb.com/flops-bay/

]]>
http://idesmanb.com/2021/01/15/flops-bay-post/feed/ 1
DIY: AI-Powered Smart Scale Part 1 http://idesmanb.com/2021/01/02/diy-ai-powered-smart-scale-part-1/?utm_source=rss&utm_medium=rss&utm_campaign=diy-ai-powered-smart-scale-part-1 http://idesmanb.com/2021/01/02/diy-ai-powered-smart-scale-part-1/#respond Sat, 02 Jan 2021 19:05:41 +0000 http://idesmanb.com/?p=143 Read More »]]>

In January, 2020 looked like we (our startup) can optimize the food sector even more with an automated food waste tracking solution. I had already seen some solutions for smart scales and I though: why don’t we build our own, it should not be that hard? And indeed it isn’t, so I would like to tell you everything around it. I hope that you find the information interesting and useful.

Some usecases

Digitalisation of the stock: Keep track of the stock in the kitchen and know exactly how much and what food ingredients are available by measuring the weight of the newly bought products and classifying them via image recognition / QR codes scans.

Expiration date: Track the quality of the food by not only capturing its weight and pictures of it, but also storing its expiration date.

Evaluation of waste: This is a use case for a smart waste bin: the waste bin will be placed on the scale and every time ingredients are thrown away one can track what exactly and how much.

How it works

1. Food waste is disposed as usual in a bin and its weight is measured by the smart scale.

2a. Semi automated: The operator chooses type of food and reason for throwing food away on a touchscreen.

2b. Automated: The scale automatically recognizes the food. The operator can enter the reason for food thrown away and correct the recognition.

3. Data (images, recognitions and measurements) are surfaced in reports, answering questions like WHAT, HOW MUCH and WHY food is thrown away.

Hardware

Before getting the mentioned components I started drawing… As bad drawer as I am, you can get a rough idea where all is heading:

If you want to build the same scale you will need:

Scale Bimco В30/К60 with RS-232 porthttp://www.bimco.net/en/Platform-Scales/9/view/
Raspberry Pi 4 Computer Modell B, 2GB
RAM (a complete overkill for the scope of the project)
USB (A) to RS-232
Raspberry Pi 4 Screen Case
Raspberry Pi 7-Inch Touch Screen Display
Logitech C270 Webcam
Stainless steel merging partCustom made 🙂

Funny enougth, the stainless steel merging part was hardest for me to get, luckily my neighbour is a craftsman so I was able to get it exactly as needed.

Once I had all the parts, I could finally get my hands dirty. I installed Ubuntu on the computer and once I was able to SHH with Putty into the machine I installed NodeJS to build and run the software. There will be an application on the Raspberry, that will collect the data from the scale, add timestamps, get pictures from the webcamera and push all that data to the AWS cloud. More on that in the next post, where I show you what software I wrote, how I ran it and how I integrated the AI part.

Before merging the Raspberry with the display I still used allmost all ports of the Raspberry. One for powering with USB-C, one USB for the web camera, one for the RS232 to USB cable that streamed the data from the scale and an Ethernet port connecting the Raspberry to my router:

Soon my next blog post for the software will follow. The link to the post will appear here.

]]>
http://idesmanb.com/2021/01/02/diy-ai-powered-smart-scale-part-1/feed/ 0
Day trip tip in Bavaria #1: Walchensee http://idesmanb.com/2020/12/27/1-day-trip-tip-in-bavaria-1-walchensee/?utm_source=rss&utm_medium=rss&utm_campaign=1-day-trip-tip-in-bavaria-1-walchensee http://idesmanb.com/2020/12/27/1-day-trip-tip-in-bavaria-1-walchensee/#respond Sun, 27 Dec 2020 20:19:05 +0000 http://idesmanb.com/?p=123 Read More »]]> In Bavaria, you can always escape the city and visit the Alps and beautiful lakes, only one – two hours away from Munich. With a notable amount of lakes, around 100, there are plenty of trips one can take and still arrive for a beer ot two in Augustiner Bräu in the evening.

One such trip tip destination that I can give you is Walchensee: with no doubts one of the most beautiful lakes in Bavaria. It is located in the south from Munich, 15 minutes by car from Kochel. There is a limited possibility to park sideways of the road and there is a parking lot near by.

Walchensee has crystal clear blue waters and rocky small beaches. I visited in in April and it offered some really nice views, warm weather and not very cold water. Definitely worth a visit!

]]>
http://idesmanb.com/2020/12/27/1-day-trip-tip-in-bavaria-1-walchensee/feed/ 0
Remote Rhapsody: combine a smart speaker and virtual reality http://idesmanb.com/2020/12/24/remote-rhapsody-combine-a-smart-speaker-and-virtual-reality/?utm_source=rss&utm_medium=rss&utm_campaign=remote-rhapsody-combine-a-smart-speaker-and-virtual-reality http://idesmanb.com/2020/12/24/remote-rhapsody-combine-a-smart-speaker-and-virtual-reality/#comments Thu, 24 Dec 2020 23:18:00 +0000 http://idesmanb.com/?p=41 Read More »]]> I had few more days off that I had to take by the end of the year. At the end of 2020 with a partial lockdown in Germany means no travelling and not much socializing, so I decided to go for an online activity and try out something new.

Lately, I wanted to start something with AR/VR: an unknown field for me, that gives big promises, from “everything is possible within a virtual world” to a rapid market growth in the next years.

Also, Deutsche Telekom, together with Hubraum and BeMyApp announced a hackathon for their smart speaker: a fast follower of Google and Alexa that has its own virtual assistant AI called Magenta.

The hackathon was about writing a skill for the speaker and publishing it using their voice skill SDK: https://github.com/telekom/voice-skill-sdk

I was curious to attend but did not wanted to develop a skill that just tells random jokes: we have seen enough of them and we all know that they work 😊 I wanted to really challenge myself here (and my teammate) and combine the smart speaker with VR.

The idea is simple: let us build a virtual teacher for the children, that will engage and teach them, especially during lockdowns and school closings. That would enable children to:

  • Answer questions, curated by the teachers
  • Be in the same virtual room with their friends
  • Make breaks with healthy exercises

The virtual teacher would use the Magenta smart speaker as an input / output device and the child will wear VR goggles for a seamless experience (or use any TV screen / Laptop or Tablet). The smart speaker will do the heavy lifting for text-to-speech and speech-to-text as well as the Natural Language Processing part.

Surprisingly, the whole hardware (speaker and VR goggles) for that is very affordable: just below 30 Euros: 21 Euros for the speaker and you can pick between or for the VR part.

For the scope of the hackathon we implemented the following dialog:

Parent/Child: Hello Magenta, start the virtual teacher!
V teacher says: Hello Sara, what do you want to do today? Algebra or Geometry? I can also show you some warm up exercises.
Child says: Algebra!
V teacher says: Alright! What is the sum of 3 and 1?
Child says: This makes 4!
V teacher says: Very good. Then I have another question: how much is it if you subtract 3 from 7?
Child says: 2!
V teacher says: Unfortunately that’s not true. Let’s try again. What do you get if you subtract 3 from 7?
Child says: 4!
V teacher says: Very good. Then I have another question: How much does it makes if you subtract 12 from 34?
Child says: Change game!
V teacher says: Alright, let’s do some geometry. What is this figure? (shows a triangle)
Child says: triangle!
V teacher says: Very good. Let’s try one more! Do you know what is that figure?
Parents/Child: Exit game!

Implementing that conversation turned out to be not a trivial task: usually when you interact with a smart speaker you would ask something and the answer would end the conversation. As you can see from the example above, here we have a dialog where the Virtual Teacher proactively asks for input in an unguided conversation. We dealed with that by attaching a micro service to the skill, isolating the conversation logic in separate serverless functions and keeping state for the skill.

Further, we needed a basic virtual class room with a whiteboard and a virtual human. A week ago, I noticed that AWS has a service called Sumerian, for everyone who wants to build AR/VR apps with virtual people inside that have decent gestures and talking capabilities. While Sumerian comes with few ready assets, you can easily import more in .fbx and .obj format. So the virtual class room took few hours until fully ready for showcasing it. I highly recommend watching the Sumerian tutorials for a quick start here

The whiteboard in the virtual class room is an “3D HTML” element and can render “normal” HTML like a browser inside the virtual reality. So we also had to develop a basic web application and embed it in the whiteboard.

To create a skill for the Magenta smart speaker, one can simply use python which makes the process very easy for newcomers like us: just import some libraries, write a regex that starts your skill, host and deploy your skill in the Skill Development Portal and you can already have a conversation with your smart speaker.

Once we had all building blocks in place, we connected them with Lambda functions and APIs and our solution was ready:

To wrap up, this hackathon was a really nice experience where I learned a lot and I was able to experiment with two great technologies: smart speakers and VR and the endless usecases around them. The Magenta smart speaker and the skill development platform turned out to have a shallow learning curve and I can’t wait to see more skills for the speaker soon. Our solution won 3rd place and we both received some nice Christmas presents 🙂

Repositories:


]]>
http://idesmanb.com/2020/12/24/remote-rhapsody-combine-a-smart-speaker-and-virtual-reality/feed/ 1
MiniHack 2: what Morse Code and Caps Lock have in common http://idesmanb.com/2020/12/19/mini-hack/?utm_source=rss&utm_medium=rss&utm_campaign=mini-hack http://idesmanb.com/2020/12/19/mini-hack/#respond Sat, 19 Dec 2020 20:32:44 +0000 http://idesmanb.com/?p=25 Read More »]]> I only write about MiniHack 2 as unfortunately I missed MiniHack 1 and most probably will miss MiniHack 3. MiniHack (https://www.meetup.com/mini-hack/) is a nice experience for somebody who has a vacation, but is not able to travel or see friends, because of Corona restrictions. Also, for people who want to have fun meeting others and solving puzzles. So, I brought my usual activity from my workdays in my calm evening: hacking. “Why are you not programming in your workdays?”, you might ask – well according to Google, hacking happens during “an event in which a large number of people meet to engage in collaborative computer programming” – and that is what I mostly do at work. And when I write code alone, I would rather call it side hustling 😉

We were randomly assigned in groups of up to 4 people and given the task: create a game with no graphics. That means we shouldn’t draw anything on the screen, even using symbols in the console. We could, however, take input and output words to the console, output sound, make an IOT device turn on/ off etc. Also, we had around 90 minutes to produce something that works.

After short intro I was assigned to the team Orange and we started brainstorming. My idea was to use the flashlight you would find next to the web camera of any decent laptop and make it blink in the Morse Code (https://en.wikipedia.org/wiki/Morse_code). That turned out to be quite tricky, because of manufacturer and privacy stuff and so we went for the small light some computers have on their Caps Lock button. Programmatically you can turn Caps Lock on and off and that is all we did.

Our program will greet you in the console, blink the Caps Lock light and ask you to decode the Morse Code message and write it back to the console. If you succeed it will continue asking for more words, gradually increasing the speed of the blinking light.

Here is our code. I decided to leave it as is, so you can improve it as you want.

//imports...

namespace ConsoleApp29
{

    class Program
    {
        [DllImport("user32.dll")]
        static extern void keybd_event(byte bVk, byte bScan, uint dwFlags, UIntPtr dwExtraInfo);

        static void Main(string[] args)
        {
            var words = new Dictionary()
            {
                ["DAVID"] = "-.. .- ...- .. -..",
                ["BORIS"] = "-... --- .-. .. ...",
                ["GAVIN"] = "--. .- ...- .. -.",
                ["NUH"] = "-. ..- ....",
                ["MILES"] = "-- .. .-.. . ...",
            };

            var random = new Random();
            var level = 1;

            Console.WriteLine("Please enter a word ... or type EXIT to leave the program");
            var userWord = string.Empty;

            while (userWord.ToUpper() != "EXIT")
            {
                var result = random.Next(0, words.Count());
                var word = words.Keys.ElementAt(result);
                var code = words[word];

                StrToBlinkConverter(code, level);

                userWord = Console.ReadLine();

                if(userWord.Equals(word, StringComparison.InvariantCultureIgnoreCase))
                {
                    Console.WriteLine("Congrats! You got the next word!");
                    level++;
                }
                else
                {
                    Console.WriteLine("That was wrong. Keep practising :)");
                }
            }
        }

        private static void TurnOff(double seconds)
        {
            const int KEYEVENTF_EXTENDEDKEY = 0x1;
            const int KEYEVENTF_KEYUP = 0x2;
            keybd_event(0x14, 0x45, KEYEVENTF_EXTENDEDKEY, (UIntPtr)0);
            keybd_event(0x14, 0x45, KEYEVENTF_EXTENDEDKEY | KEYEVENTF_KEYUP,
            (UIntPtr)0);
            Thread.Sleep(TimeSpan.FromSeconds(seconds));
        }

        private static void TurnOn(double seconds)
        {
            const int KEYEVENTF_EXTENDEDKEY = 0x1;
            const int KEYEVENTF_KEYUP = 0x1;
            keybd_event(0x14, 0x45, KEYEVENTF_EXTENDEDKEY, (UIntPtr)0);
            keybd_event(0x14, 0x45, KEYEVENTF_EXTENDEDKEY | KEYEVENTF_KEYUP,
            (UIntPtr)0);
            Thread.Sleep(TimeSpan.FromSeconds(seconds));

        }

        private static void StrToBlinkConverter(string word, int level)
        {
            foreach (var letter in word)
            {
                switch (letter)
                {
                    case '.':
                        TurnOn(2 / (double)level);
                        TurnOff(1 / (double)level);
                        break;
                    case '-':
                        TurnOn(4 / (double)level);
                        TurnOff(1 / (double)level);
                        break;
                    default:
                        TurnOff(1 / (double)level);
                        break;
                }
            }

            Console.Write("? ");
        }
    }
}

We actually won this edition of MiniHack, and I got this nice present:


Many thanks to Paul Michaels for organizing it.


]]>
http://idesmanb.com/2020/12/19/mini-hack/feed/ 0