Roundup 13

my almost biweekly random notes

Numbers:
- There is 42 tons of litter is collected from Amsterdam canals per year.
TIL
- If you have google wifi (aka nest wifi), if you open a port, you are opening to the world publicly. Careful.
- Benford’s Law: Briefly explained, Benford’s Law maintains that the numeral 1 will be the leading digit in a genuine data set of numbers 30.1% of the time; the numeral 2 will be the leading digit 17.6% of the time; and each subsequent numeral, 3 through 9, will be the leading digit with decreasing frequency.
How to get the redirect chain triggered from a URL with the URLs used on the way till the final page URL:

wget https://hakanu.net 2>&1 | grep Location

Wordpress SSL problem with Cloudflare
- Turn on flexible SSL from cloudflare (full SSL doesn’t work for some reason, cloudflare serves 501 due to server error)
- Make your site_url https://mysitename, home is not so important. You can do that from phpmyadmin or from wp-config.php.
- Now you can not login to wp-admin, either:
  - Loops back to login page with no errorws even with debug logs enabled.
  - Says “You are not allowed see this page”, hell i am
  - Or nothing loads, Chrome/Firefox says there is an infinite number of redirects.
    - Check the redirect chain of a url like this in linux (ubuntu):
    wget https://hakanu.net 2>&1 | grep Location:
  - Or if you enable SSL but don’t change the site_url from Settings in the Wordpress Dashboard, your images etc will be served through http under https site which causes mixed content error and in the modern browsers those static items stop serving.
    - Or From wp-config.php:
    define('WP_HOME','https://yourdomain.com'); define('WP_SITEURL','https://yourdomain.com');
  - Now the most important part, add this AT THE TOP OF THE FILE otherwise it won’t be effective (or half effective lol):
```
if ( isset( $_SERVER['HTTP_X_FORWARDED_PROTO'] ) && 'https' == $_SERVER['HTTP_X_FORWARDED_PROTO'] ) {
    $_SERVER['HTTPS'] = 'on';
}
```
How to send large videos from whatsapp web? Overcoming whatsapp’s 64 mb file size limit. Answer is easy: decrease the size of the video.
- You can use your favorite video editor like Windows 10’s native Movies app or Da Vinci Resolve. I usually use ffmpeg because makes me feel fancier. Kidding aside, it’s way more customizable and solid than any GUI tool out there and I have tons of scripts written already.
- The most effective way to work around is using ffmpeg to decrease bitrate. It significantly decreases the size of the video.
- First let’s understand the bitrate:
  - ffmpeg -i $1
- Now let’s decrease it to 6000k. -strict -2 is needed otherwise ffmpeg complains about -b being ambigious. bash ffmpeg -i $1 -vcodec h264 -b 6000k -strict -2 $1_smaller_`date +%Y%m%d_%H%M`.mp4
- And to note (Today I learned), don’t use -acodec mp3 or -acodec mp2 because iPhone users won’t be able to hear the sound of your video. Apparently iOS uses aac for encoding sound. Android plays all of them nicely though.
  - Best way to fix that problem is not putting a -acodec parameter. Probably original input file is using aac and ffmpeg’s generated file will have same audio codec.
Get database statistics of a mongodb database:

mongo "$MONGO_CONNECTION_URL"
db.runCommand( { dbStats: 1})

Serving python web servers at production in high scale:
- Found awesome pythonic framework which is adding production level service capabilities to your server: waitress-serve.
  - pip install waitress
  - Bonus: You can use it with web.py, flask, or anything else as long as it’s compatible with WSGI.
  - Example web.py production integration:
```
  import waitress
  waitress.serve()
```
  - Example flask production integration:
```
  import waitress
  waitress.serve()
```
- Even better alternative is gunicorn for which I’m going to write a separate blog post.

How to check how much RAM redis is using?

  redis-cli
  redis 127.0.0.1:6379> info
  # Check out used_memory_human

How to increase open file limit for redis?

  ulimit -n   # Should be somewhere around 1024.
  sudo ulimit -n 65536
  ulimit -n  # Should be updated.

How to install a deb package in debian/ubuntu without apt-get:
```
  sudo dpkg -i /e/Downloads/duf_0.3.1_linux_amd64.deb
```
How to pinpoint what’s slow with your flask app
- working fine in local but slow in production?
- You can use werkzeug’s default profiler: ModuleNotFoundError: No module named 'werkzeug.contrib'
  - There are some recommendations to downgrade werkzeug because profiler was in contrib before. Please please don’t do that. If you don’t have a good reason, never use old versions of the libraries.
  - Solution: This is from old version, use new one
```
  # pip install werkzeug
  from werkzeug.middleware.profiler import ProfilerMiddleware
  app = ProfilerMiddleware(app)
```
Let other command line params to be passed through argparse. args, unknown = parser.parse_known_args()
- eg. gunicorn: error: unrecognized arguments: --bind 0.0.0.0:8081 gae:app
I was getting this error while deploying my mongodb atlas hosted db backed python flask app on google app engine and heroku: pymongo.errors.ServerSelectionTimeoutError: connection closed,connection closed,connection closed heroku
- Full read about the problem with example code is here
- The problem lies under the IP whitelisting of mongodb atlas.
  - One solution: add 0.0.0.0/0 to iptable of mongodb atlas so that it’s open to whole world which is quite unsafe.
  - instead you can get all heroku ips with this command and add to mongodb atlas from admin screen.
```
  HEROKU_REGION=eu; sudo apt -qqy install curl jq 2>/dev/null 1>/dev/null; heroku regions --json 2>/dev/null | jq ".[] | select(.name==\"$HEROKU_REGION\") | .provider.region" | (REGION=$(cat); curl -s https://ip-ranges.amazonaws.com/ip-ranges.json |  jq ".prefixes[] | select(.region==$REGION) | .ip_prefix") > ip.txt
```
  - Add the output to the mongodb IP whitelist table. (probably one by one, check out my blog post about automating whitelisting of large number of IPs.)