Jekyll2024-02-20T07:47:04+00:00http://hakanu.net/feed.xml{“title”: “Who let the dogs out”}Some random hacker writes random stuff on software development, frameworks, entrepreneurship, geeky stuff, startups, python, serverless, web development, big data, machine learning, data mining, parallel processing, GPUs etc.
Enable x265 (HEVC) hardware decoding in Raspberry Pi 42024-02-20T00:00:00+00:002024-02-20T00:00:00+00:00http://hakanu.net/raspberrypi/2024/02/20/enable-x265-hevc-hardware-decoding-in-raspberry-pi-4<p>My Emby Server struggles a lot during playing x265 formatted videos, it gets stutters every 3-4 seconds. Very annoying. I resolved like this:</p>
<ul>
<li>Edit <code class="highlighter-rouge">/boot/config.txt</code>: sudo vim /boot/config.txt</li>
<li>Add this line to the end</li>
</ul>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>dtoverlay=rpivid-v4l2
</code></pre></div></div>
<p>(More details about <a href="https://www.reddit.com/r/jellyfin/comments/wpaxl8/raspberry_pi_4_using_v4l2/">v4l2</a>)</p>
<ul>
<li>Save the file.</li>
<li>Reboot the pi: <code class="highlighter-rouge">sudo reboot now</code></li>
</ul>
<p>If this doesn’t solve your problem, check out the HEVC support in your browser. The client consuming emby content should also be compatible.</p>My Emby Server struggles a lot during playing x265 formatted videos, it gets stutters every 3-4 seconds. Very annoying. I resolved like this:Solving “Error: TRIGGER_PAYLOAD_TOO_LARGE: This request would cause a function payload exceeding the maximum size allowed” in firebase realtime db2023-12-24T00:00:00+00:002023-12-24T00:00:00+00:00http://hakanu.net/firebase/2023/12/24/solving-error-trigger-payload-too-large-this-request-would-cause-a-function-payload-exceeding-the-maximum-size-allowed-in-firebase-realtime-db<p>I was using this piece and it stopped working in one night:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="n">db</span><span class="o">.</span><span class="n">reference</span><span class="p">(</span><span class="s">'foo/some_key'</span><span class="p">)</span><span class="o">.</span><span class="n">delete</span><span class="p">()</span>
</code></pre></div></div>
<p>I wrote the code above in different variations, nothing helped.
Writing it here so that I won’t forget. Eventually the solution was very easy, I had an inactive firebase function listening to firebase realtime database writes and I just deleted that function. This thing got resolved.</p>
<p>The inspo is from this old post:</p>
<p>https://stackoverflow.com/questions/50513374/firebase-realtime-database-currently-gives-trigger-payload-too-large-error</p>
<p>Hope saves someones less hours than me.</p>I was using this piece and it stopped working in one night:Solving “Invalid JWT: Token must be a short-lived token” error while using firebase realtime db on WSL2023-12-24T00:00:00+00:002023-12-24T00:00:00+00:00http://hakanu.net/wsl/2023/12/24/solving-invalid-jwt-token-must-be-a-short-lived-token-error-while-using-firebase-realtime-db-on-wsl<p>I keep hitting this in my Windows laptop with WSL (Windows Subsystem for Linux).<br />
This is the full error:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>google.auth.exceptions.RefreshError: <span class="o">(</span><span class="s1">'invalid_grant: Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim.'</span>, <span class="o">{</span><span class="s1">'error'</span>: <span class="s1">'invalid_grant'</span>, <span class="s1">'error_description'</span>: <span class="s1">'Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim.'</span><span class="o">})</span>
</code></pre></div></div>
<p>Check out the date: <code class="highlighter-rouge">date</code> => Probably will output something weird.</p>
<p>Sync it again (needs sudo):</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>hwclock <span class="nt">-s</span>
</code></pre></div></div>
<p>Check out date date again: <code class="highlighter-rouge">date</code> => This time it should show correct timezone. if not, there are more commands to run I stole from internet.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>timedatectl set-timezone Europe/Amsterdam
<span class="nb">date</span>
<span class="c"># if this doesn't work</span>
<span class="nb">sudo </span>apt-get update
<span class="nb">sudo </span>apt-get <span class="nb">install</span> <span class="nt">-y</span> ntpdate
<span class="nb">sudo </span>ntpdate pool.ntp.org
</code></pre></div></div>I keep hitting this in my Windows laptop with WSL (Windows Subsystem for Linux). This is the full error:Fixing mongodb start error code 14/n/a2023-08-15T00:00:00+00:002023-08-15T00:00:00+00:00http://hakanu.net/mongodb/2023/08/15/fixing-mongodb-start-error-code-14-n-a<p>Stupid errors today.</p>
<p>My local mongodb which has been running for multiple weeks decided not to start.</p>
<p>First I need to remove MONGODB_CONFIG_OVERRIDE_NOFORK env variable.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>vim /usr/lib/systemd/system/mongod.service
<span class="c"># Convert this line:</span>
<span class="c"># Environment="MONGODB_CONFIG_OVERRIDE_NOFORK=1"</span>
<span class="c"># To This line:</span>
<span class="c"># Environment="MONGODB_CONFIG_OVERRIDE_NOFORK=0"</span>
</code></pre></div></div>
<p>Then it still refuse to up itself, but this time no errors:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>systemctl start mongod
x:~<span class="nv">$ </span><span class="nb">sudo </span>systemctl status mongod
× mongod.service - MongoDB Database Server
Loaded: loaded <span class="o">(</span>/lib/systemd/system/mongod.service<span class="p">;</span> enabled<span class="p">;</span> vendor preset: enabled<span class="o">)</span>
Active: failed <span class="o">(</span>Result: exit-code<span class="o">)</span> since Mon 2023-08-14 20:33:54 CEST<span class="p">;</span> 1s ago
Docs: https://docs.mongodb.org/manual
Process: 1501 <span class="nv">ExecStart</span><span class="o">=</span>/usr/bin/mongod <span class="nt">--config</span> /etc/mongod.conf <span class="o">(</span><span class="nv">code</span><span class="o">=</span>exited, <span class="nv">status</span><span class="o">=</span>14<span class="o">)</span>
Main PID: 1501 <span class="o">(</span><span class="nv">code</span><span class="o">=</span>exited, <span class="nv">status</span><span class="o">=</span>14<span class="o">)</span>
Aug 14 20:33:54 x systemd[1]: Started MongoDB Database Server.
Aug 14 20:33:54 x systemd[1]: mongod.service: Main process exited, <span class="nv">code</span><span class="o">=</span>exited, <span class="nv">status</span><span class="o">=</span>14/n/a
Aug 14 20:33:54 x systemd[1]: mongod.service: Failed with result <span class="s1">'exit-code'</span><span class="nb">.</span>
</code></pre></div></div>
<p>Apparently it’s a <a href="https://askubuntu.com/questions/823288/mongodb-loads-but-breaks-returning-status-14">permission issue</a>:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo chown</span> <span class="nt">-R</span> mongodb:mongodb /var/lib/mongodb
<span class="nb">sudo chown </span>mongodb:mongodb /tmp/mongodb-27017.sock
<span class="nb">sudo </span>service mongod restart
</code></pre></div></div>Stupid errors today.Sqlite3 over network shared drive and database getting locked error2023-04-15T00:00:00+00:002023-04-15T00:00:00+00:00http://hakanu.net/python/2023/04/15/sqlite3-over-network-shared-drive-and-database-getting-locked-error<p>Sqlite3 apparently doesn’t like my new NAS setup. Reads are fine but writes were not working and regardless of the timeout I give it was failing with database is locked error.</p>
<p>Error I get:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>python3.9/site-packages/peewee.py", line 3221, in execute_sql cursor.execute(sql, params or ()
sqlite3.OperationalError: database is locked
</code></pre></div></div>
<p>I had to play with pragmas a bit. At the end what worked is</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sqlite3</span> <span class="n">mydatabase</span><span class="p">.</span><span class="n">db</span>
<span class="o">#</span> <span class="k">Write</span> <span class="n">ahead</span> <span class="n">logging</span><span class="p">:</span>
<span class="o">#</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="p">.</span><span class="n">sqlite</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">pragma</span><span class="p">.</span><span class="n">html</span><span class="o">#</span><span class="n">pragma_journal_mode</span>
<span class="n">PRAGMA</span> <span class="n">journal_mode</span><span class="o">=</span><span class="n">WAL</span><span class="p">;</span>
<span class="o">#</span> <span class="n">Made</span> <span class="n">this</span> <span class="k">to</span> <span class="n">normal</span> <span class="k">from</span> <span class="k">full</span><span class="p">.</span>
<span class="o">#</span> <span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">www</span><span class="p">.</span><span class="n">sqlite</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">pragma</span><span class="p">.</span><span class="n">html</span><span class="o">#</span><span class="n">pragma_synchronous</span>
<span class="n">PRAGMA</span> <span class="n">synchronous</span><span class="o">=</span><span class="n">normal</span><span class="p">;</span>
</code></pre></div></div>
<p>You can obviously use DB Browser sqlite UI:</p>
<p><img src="https://devdala.files.wordpress.com/2023/04/screenshot-2023-04-15-010508.png" alt="" /></p>
<p>If this doesn’t work either, unmount the network drive and mount it like this with <code class="highlighter-rouge">nobrl</code> option which prevents byte range locking but this comes with disadvantage of likelihood of database corruptions</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>mount <span class="nt">-t</span> cifs //192.168.1.100/MY_SHARE_NAME /mnt/MY_MOUNT_FOLDER <span class="nt">-o</span> <span class="nv">username</span><span class="o">=</span>my_user,password<span class="o">=</span>my_pass,uid<span class="o">=</span><span class="k">$(</span><span class="nb">id</span> <span class="nt">-u</span><span class="k">)</span>,gid<span class="o">=</span><span class="k">$(</span><span class="nb">id</span> <span class="nt">-g</span><span class="k">)</span>,nobrl
</code></pre></div></div>Sqlite3 apparently doesn’t like my new NAS setup. Reads are fine but writes were not working and regardless of the timeout I give it was failing with database is locked error.Fresh Debian install and username not in sudoers file2023-03-29T00:00:00+00:002023-03-29T00:00:00+00:00http://hakanu.net/linux/2023/03/29/fresh-debian-install-and-username-not-in-sudoers-file<p>Note for my future self to solve this quickly. Because there are really long tutorials on this.</p>
<p>I have got a fresh debian 11 installation yet, I can not do sudo apt install anything because of this error. You can not modify things like usermod because it needs sudo privileges.</p>
<p>Here is the quick solution:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>su root
vim /etc/sudoers
<span class="c"># Into the file add your username:</span>
your_username <span class="nv">ALL</span><span class="o">=(</span>ALL<span class="o">)</span> ALL
<span class="c"># Save and exit, boom</span>
</code></pre></div></div>Note for my future self to solve this quickly. Because there are really long tutorials on this.SSH password connection rejected after OpenMediaVault install2023-03-26T00:00:00+00:002023-03-26T00:00:00+00:00http://hakanu.net/linux/2023/03/26/ssh-password-connection-rejected-after-openmediavault-install<p>This is so stupid but took my 1 hour to figure out.</p>
<p>I installed Open Media Vault to create some raids and home NAS on a debian machine. However, after the installation it started to reject my ssh connection to the machine.</p>
<p>Things to look for:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>vim /etc/ssh/sshd_config
<span class="c"># Add this line: AllowUsers your_username</span>
<span class="c"># Change this line: PermitRootLogin yes</span>
<span class="c"># Restart the service.</span>
<span class="nb">sudo </span>service ssh restart
<span class="c"># Mine is not fixed so I had to put my username into ssh group</span>
<span class="nb">sudo </span>usermod <span class="nt">-aG</span> ssh your_username
</code></pre></div></div>This is so stupid but took my 1 hour to figure out.Changing default port and host of Resilio Sync2023-02-03T00:00:00+00:002023-02-03T00:00:00+00:00http://hakanu.net/sync/2023/02/03/changing-default-port-and-host-of-resilio-sync<p><img src="https://devdala.files.wordpress.com/2023/02/screenshot_20230203_082422.png" alt="" /></p>
<p>Why does it have to be so hard. Good docs but there is no good documentation about the default configuration directory. For linux it’s <code class="highlighter-rouge">/etc/resilio-sync/config.json</code></p>
<p>If you want remote server to be accessed remotely you gotta bind 0.0.0.0 and default resilio port is 8888.</p>
<p>Open it up:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>vim /etc/resilio-sync/config.json
</code></pre></div></div>
<p>Add this snippet:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="s2">"use_gui"</span><span class="p">:</span><span class="w"> </span><span class="kc">false</span><span class="p">,</span><span class="w">
</span><span class="s2">"webui"</span><span class="w"> </span><span class="p">:</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="s2">"listen"</span><span class="w"> </span><span class="p">:</span><span class="w"> </span><span class="s2">"0.0.0.0:9999"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>If you haven’t installed, it’s like this for linux (or raspberry pi):</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">echo</span> <span class="s2">"deb http://linux-packages.resilio.com/resilio-sync/deb resilio-sync non-free"</span> | <span class="nb">sudo tee</span> /etc/apt/sources.list.d/resilio-sync.list
curl <span class="nt">-L</span> https://linux-packages.resilio.com/resilio-sync/key.asc | <span class="nb">sudo </span>apt-key add
<span class="nb">sudo </span>dpkg <span class="nt">--add-architecture</span> armel <span class="o">&&</span> <span class="nb">sudo </span>apt-get update <span class="o">&&</span> <span class="nb">sudo </span>apt-get <span class="nb">install </span>resilio-sync:armel
<span class="nb">sudo </span>apt-get update
<span class="nb">sudo </span>apt-get <span class="nb">install </span>resilio-sync
</code></pre></div></div>Weird WSL 2 gotchas - systemd, ssh server, directory permissions, space compaction etc2023-02-03T00:00:00+00:002023-02-03T00:00:00+00:00http://hakanu.net/wsl/2023/02/03/weird-wsl-2-gotchas-systemd-ssh-server-directory-permissions-space-compaction-etc<p>As much as I love WSL it comes with weird quirks you need to work around. eg. you can not git clone a repo in another directory than C: (I guess) unless you do what I tell you below lol. If there are easier ways, happy to hear.</p>
<h2 id="move-wsl-between-drives-to-open-up-space">Move WSL between drives to open up space</h2>
<pre><code class="language-bat">wsl --list -v
wsl --list -v
wsl -t Ubuntu
wsl --export Ubuntu "D:\wsl_export\ubuntu-ex.tar"
wsl --unregister Ubuntu
wsl --import Ubuntu "D:\WSL\import" "D:\WSL\ubuntu-ex.tar"
</code></pre>
<h2 id="shut-down-wsl-instance">Shut down WSL instance</h2>
<p>This is useful when you update /etc/wsl.conf</p>
<pre><code class="language-bat">wsl --shutdown
</code></pre>
<h2 id="find-wsl-folder">Find WSL folder:</h2>
<p><code class="highlighter-rouge">\HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Lxss</code></p>
<h2 id="space-up">Space up</h2>
<p>WSL is weird because the more you use space within WSL, eg. apt install things and then remove things. It doesn’t return the space back to the host Windows 11 filesystem back. Instead you gotta do the compaction yourself.</p>
<p>Save this to compact-disk.txt</p>
<pre><code class="language-bat">select vdisk file="C:\Users\$whoami\AppData\Local\Packages\CanonicalGroupLimited.Ubuntu_79rhkp1fndgsc\LocalState\ext4.vhdx"
attach vdisk readonly
compact vdisk
detach vdisk
</code></pre>
<p>Run this
diskpart /s <SAVED_SCRIPT_FOLDER_PATH>\compact-disk.txt</SAVED_SCRIPT_FOLDER_PATH></p>
<h2 id="increase-the-ram-allocated-for-wsl">Increase the ram allocated for WSL</h2>
<p>WSL by default is bound to 8 or half of the host’s RAM as far as I understand. I go yolo here because I’m trying to run stable diffusion models.</p>
<p>Add .wslconfig under C:\User$whoami</p>
<pre><code class="language-bat">[wsl2]
memory=120GB # Limits VM memory in WSL 2 to 128 GB
</code></pre>
<h2 id="enable-ssh-server">Enable SSH Server</h2>
<p>This is different from Windows 11’s default SSH support. This is a workaround again.
Windows itself has a nice ssh access support but when you access to windows from ssh you can not simply <code class="highlighter-rouge">bash</code> into WSL. Because it’s not working yet if you installed WSL through Microsoft Store which is the officially recommended way to get the proper updates. Well, hmm. So your best bet is either uninstalling WSL and installing through non MS Store or ssh-ing into the WSL itself directly. I chose the second one here however, in order for this to work you gotta keep the WSL on when you boot windows. Annoying while on holidays.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt <span class="nb">install </span>openssh-server
<span class="nb">sudo</span> /usr/sbin/service ssh start
ifconfig
<span class="nb">hostname
whoami
</span>ssh hakanu@172.18.211.124
</code></pre></div></div>
<h2 id="enable-systemd">Enable systemd</h2>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>vim /etc/wsl.conf
</code></pre></div></div>
<p>Add these lines</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[boot]
systemd=true
</code></pre></div></div>
<h2 id="resolve-git-issues-operation-not-permitted">Resolve git issues: Operation not permitted</h2>
<p>When you git clone, it gives this stupid error, smt like this:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>.git/config.lock failed: Operation not permitted
fatal: could not set 'core.filemode' to 'false'
</code></pre></div></div>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>vim /etc/wsl.conf
</code></pre></div></div>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[automount]
options = "metadata"
</code></pre></div></div>
<p>From cmd.exe, reboot WSL: <code class="highlighter-rouge">wsl --shutdown</code></p>
<p>Voila.</p>As much as I love WSL it comes with weird quirks you need to work around. eg. you can not git clone a repo in another directory than C: (I guess) unless you do what I tell you below lol. If there are easier ways, happy to hear.Make bootable flash drive from mac for ISO files2022-09-24T00:00:00+00:002022-09-24T00:00:00+00:00http://hakanu.net/linux/2022/09/24/make-bootable-flash-drive-from-mac-for-iso-files<ul>
<li>Download the iso file somewhere online. In this example I will do manjaro bootable usb pen drive from xfce minimal image.</li>
<li>Fire up the terminal, run these commands.</li>
</ul>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># Find which one is your flash drive</span>
diskutil list
<span class="c"># In my case it was /dev/disk3</span>
diskutil secureErase 1 /dev/disk3
<span class="nb">cd </span>Downloads/
<span class="nb">ls
</span>hdiutil convert manjaro-xfce-21.3.7-minimal-220816-linux515.iso <span class="nt">-format</span> UDRW <span class="nt">-o</span> manjaro-xfce-21.3.7-minimal-220816-linux515.img
diskutil unmountDisk /dev/disk3
<span class="nb">sudo dd </span><span class="k">if</span><span class="o">=</span>manjaro-xfce-21.3.7-minimal-220816-linux515.img.dmg <span class="nv">of</span><span class="o">=</span>/dev/disk3 <span class="nv">bs</span><span class="o">=</span>1m
</code></pre></div></div>Download the iso file somewhere online. In this example I will do manjaro bootable usb pen drive from xfce minimal image. Fire up the terminal, run these commands.