Better way to copy demo code from docs.python.org

I read a lot of code in Python docs. So there comes so much demo code to run on my own way with modifications. So I have to copy those demo, but its format is like:

>>> transposed = []
>>> for i in range(4):
... # the following 3 lines implement the nested listcomp
... transposed_row = []
... for row in matrix:
... transposed_row.append(row[i])
... transposed.append(transposed_row)

Needs to be turned to:

transposed = []
for i in range(4):
# the following 3 lines implement the nested listcomp
transposed_row = []
for row in matrix:
transposed_row.append(row[i])
transposed.append(transposed_row)

This leads to a fix of removing “>>>” and “…” and also indentations and it takes time.

So I decided to create a Chrome extension to help me out. This is quite simple, that when I copy the demo code, it removes those redundancies and be ready to be run.

I use this regex to catch it:

/^>>> |^... |^...$/gm

The rest is so simple, just several minutes for a simple extension.

document.addEventListener('copy', function(e) {
    var selectedText = window.getSelection().toString();

    if (~selectedText.indexOf('>>>')) {
        var fixedCode = selectedText.replace(/^>>> |^... |^...$/gm, '');
        e.clipboardData.setData('text/plain', fixedCode);
        e.preventDefault();
    }
});

Source code: https://github.com/ducfilan/Copy-Python-docs-code-extension

Advertisements

git pull error: “error: insufficient permission for adding an object to repository database .git/objects”

Today, when I was pulling changes from a remote repository, the error busted out:

error: insufficient permission for adding an object to repository database .git/objects

The error message indicates the problem itself, current user does not have insufficient permission to add objects to .git/objects folder.

If you’re familiar with Linux ownership and permission, it’s so easy for you. In case of not, read the following topic, it’s really interesting though: Linux Ownership and Permissions

So, we need to grant permission to the folder. We have 2 approaches:

1. Change owner of the folder and its sub-folders to current user

$chown [current-user-name]:[group-name] .git/objects -R

In my case:

$chown ducfilan:root .git/objects -R

Note: current username can be gotten from:

$echo $USER

2. Chmod the folder and its sub-folders to grant write permission to the current user

If you execute the below command, you will see your permission to the objects folder:

$ll .git | grep objects
$ll .git/objects

If you faced with the mentioned problem, you will see that you don’t have permission to that folder. Depends on your decision, you can grant which permissions to the folder you want. Maybe:

$chmod 766 .git/objects -R

Now you are ready to pull it down.

P/s: Personally, I recommend the first approach, it’s a natural and secure way.

Docker completion for Zsh with Prezto

I am currently using Prezto as the configuration framework for Zsh. Having completion while working with Docker on Terminal can boost our productivity a little bit to focus on what’s more important. So let’s get it configured.

~/.z/m/c/e/src  curl -fLo ~/.zprezto/modules/completion/external/src/_docker https://raw.githubusercontent.com/docker/cli/master/contrib/completion/zsh/_docker
~/.z/m/c/e/src  curl -fLo ~/.zprezto/modules/completion/external/src/_docker-compose https://raw.githubusercontent.com/docker/compose/master/contrib/completion/zsh/_docker-compose
~/.z/m/c/e/src  exec $SHELL -l

Now check if it works.

2017-08-28_101501

How to setup an SFTP server with RSA public key authentication mechanism

Why SFTP over FTP? The reason is visualized in its name: “S”, that means Security. Using SSH will employ a client-server model to authenticate two parties and encrypt the data between them. This topic will guide you through how to setup an SFTP authentication mechanism using public key cryptography, the working OS is CentOS version 6.9. Let’s get started!

1. Make sure ssh and ssh-server are installed

user@localhost:$which ssh
/usr/bin/ssh

2. Create a new user and a new group in server

To be easier and clarified in permission managing, we should create a separated group for SFTP and add the corresponding user to the group. To acquire this, using the following groupadd and useradd command under root:

user@localhost:$sudo groupadd sftp_users 
user@localhost:$sudo useradd sftp_user1 
user@localhost:$sudo passwd sftp_user1
user@localhost:$sudo usermod -G sftp_users sftp_user1

3. Generate RSA public and private key

Let’s make a recall how public key cryptography works. This link for detail. In short, SSH key pairs can be used to authenticate a client to a server. The client creates a key pair and then uploads the public key to any remote server it wishes to access. This is placed in a file called authorized_keys within the ~/.ssh directory in the user account’s home directory on the remote server.

If you’re under *nix based OS, you can use ssh-keygen to generate keys as being described below. Otherwise, for Windows, you can use PuTTY, you can refer this article to know how to process the generating.

user@localhost:$ssh-keygen -t rsa -f sftp_rsa

Then, copy the public key to the server within the ~/.ssh folder (corresponding to which user will be authenticated).

user@localhost:$cd /home/sftp_user1/ 
user@localhost:$mkdir .ssh # In case of no .ssh folder inside
user@localhost:$ls -a
...
.ssh
...
user@localhost:$cd .ssh # <= Copy the public key to this folder

4. Correct permissions and owner

user@localhost:$cd /home/sftp_user1/
user@localhost:$chmod 700 .ssh 
user@localhost:$chown sftp_user1:sftp_user1 .ssh 
user@localhost:$cd .ssh mv sftp_rsa.pub authorized_keys
user@localhost:$chmod 600 authorized_keys
user@localhost:$chown sftp_user1:sftp_user1 authorized_keys

5. Change SSH configurations

user@localhost:$vi /etc/ssh/sshd_config

Check the following configurations (uncomment these settings by removing # if needed):

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
PasswordAuthentication no

6. Restart the service

 user@localhost:$service sshd restart

Now you can check if it works by using any program that supports SFTP.

Reset hidden fields in HTML form using Javascript

As normal, the clear button is intended for clearing user input, and since hidden inputs are not directly accessible by the user, it doesn’t make sense to allow the user to reset the hidden input’s value. So, the clear form action remains the hidden input values if it varies.

But, in some rare cases, we still need to reset the value of those hidden fields.

Below is several methods I thought of.

1. Cache hidden fields to a safe place

I created a supporter module to support us caching the hidden fields in a form. This requires jQuery to work.

var hiddenResetSupporter = (function() {
    var _form;
    var _initForm = function(formSelector) {
        _form = $(formSelector);
    };
    var _fallbackHiddenFieldsWithTextbox = function() {
        _form.find('input[type=hidden]').each(function() {
            var fallbackTextbox = $('<input>')
                .prop('type', 'hidden')
                .prop('id', 'fallback_textbox_' + $(this).prop('name'))
                .val($(this).val());

            _form.after(fallbackTextbox);
        });
    };

    return {
        setup: function(formSelector) {
            _initForm(formSelector);
            _fallbackHiddenFieldsWithTextbox();
        },
        registerResetEvent: function(callback) {
            var resetPerformElement = _form.find('[type=reset]');
            if (resetPerformElement === undefined) return;

            resetPerformElement.click(function(e) {
                _form.trigger('reset');
                hiddenResetSupporter.resetHiddenFields();
                if (callback) callback();

                e.preventDefault();
            });
        },
        resetHiddenFields: function() {
            _form.find('input[type=hidden]').each(function() {
                $(this).val($('#fallback_textbox_' + $(this).prop('name')).val());
            });
        }
    }
})();

For using example, I created a jsfiddle: https://jsfiddle.net/ducfilan/dbptztsd/

2. Serialize hidden fields to sessionStorage and restore them back in time

Because sessionStorage are widely supported in almost all browsers (for example, even the fastidious bad boy IE supports it in IE versions from 8), you can have a reference here.

So in common situations, you can use sessionStorage as an option here. This is a link to know how it works.

Below is a simple module I created to illustrate how it is done, it still contains several problems of the serialization process.

var hiddenFieldSerializer = (function(){
	var _form;
	var _serializedData;
	var _keyName;

	return {
		setup: function(formSelector, keyName){
			_form = $(formSelector);
			_keyName = keyName;
			_serializedData = [];
		},
		serialize: function(keyName){
			_form.find("input[type='hidden']").each(function() {
				if ($(this).attr('name') ) {
					_serializedData.push($(this).attr('name') + ':' + $(this).val());
				}
			});
			sessionStorage.setItem(_keyName, _serializedData.join());
			return true;
		},
		deserialize: function(){
            if (!sessionStorage.getItem(_keyName)) return;

            var formData = sessionStorage.getItem(_keyName).split(',');

            $.each(formData, function(i, item) {
                var s = item.split(':');
                var elem = _form.find('[name="' + s[0] + '"]');
				elem.val(s[1]);
            });

			return true;
        }
	}
})();

So, you can refer above 2 approaches or what you think of when you need to reset hidden fields in a form.

How to access the web server in VMWare Fusion on its host machine

Currently, I am in a team developing a web application that supports IEs. My development environment runs under Centos.

Some UI problems appear on IE but not other browsers. This situation leads me to be in need of reproducing those problems on my development environment to confirm if my issues are fixed. I tried to install IE on Centos using Wine, but it doesn’t work properly.

So, I think if I could access the web server on the virtual machine in its host?

Below steps are what I did to make it works.

1. Set the Virtual Machine’s Network Adapter to Bridged:

2017-05-22_171609

2. Get the IP V4 of the virtual machine using the below command in Terminal:

ifconfig
eth1 Link encap:Ethernet HWaddr 00:0C:29:11:C1:E8 
inet addr:192.168.0.178 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe11:c1e8/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:536823 errors:0 dropped:0 overruns:0 frame:0
TX packets:241377 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 
RX bytes:537011299 (512.1 MiB) TX bytes:27427710 (26.1 MiB)

Now you can browse your VM’s localhost outside of the virtual machine (your host machine) by replacing “localhost” by “192.168.0.178“.

Absolute position table with padding

Today, I have a task of widening the tables to match 100% browser width to view more contents while keeping other elements stay the as they were.

pjimage

The problem is, I have so many tables to do (about 20 pages including that kind of table). I am too lazy to edit all of those pages. It really takes time and potential to introduce bugs.

So, I choose javascript and CSS to help me out.

Firstly, those tables have the class named “table_wide”.

We will use the CSS absolute position’s power of ignorance. We can make the table to be absolute with 100% percent width. But the problem is, it also ignores padding or margin. So our table will be stuck to the left and right edges.

Well, challenge accepted.

Absolute position ignores everything but it has top, right, bottom and left instead. However, top, right, bottom and left are not adapted to its width. So we cannot use padding properly.

We will use what we have to acquire what we want.


$(".table_wide").each(function(){
var paddingInPx = 10;
var windowWidthInPx = $(window).width();

// Calculate the width of table.
var thisWidthInPercent = 100 - (100/windowWidthInPx*2*paddingInPx);

// Create an empty div to fill up the space the absolute table occupied in the past.
$(this).after('<div style="height: ' + this.offsetHeight + 'px;"></div>');
$(this).wrap('<div style="position: absolute; width: ' + thisWidthInPercent + '%; left: ' + paddingInPx + 'px"></div>');
});

.table_wide {
    display: block;
    overflow: auto;
}

Well done! Challenge accomplished.

Jsfiddle: https://jsfiddle.net/ducfilan/p961moLd/

How to debug/inspect your mobile web page on a real mobile device (with or without real device)

When developing web apps for with mobile support, we usually use Device toolbar (Ctrl + Shift + M) on Google Chrome Developer Tools. But, the problem is that sometimes our layout or javascript code work on Device toolbar but not a real device and there is no dev tool on mobile browsers.

Luckily, Chrome Developer Tools provides us a really useful tool to debug our web page on a real android device on our desktop totally like what we do on desktop web pages.

Installation and interacting with Remote debugging are described here.

In case you don’t have an Android device in hand. You could use an Emulator. It works perfectly with me.

I use Genymotion, it’s really fast and stable.

You can use any mobile browser and be able to debug your web pages on that browser.

You can install browsers to the Genymotion Emulator by drag and drop your .apk file to the emulator screen.

Or you can install Google Play Store and get them downloaded just like a normal device.

The processes are described here.

Goodluck!

How I setup my Terminal on Linux

Today, I got a new computer in my company. So I have to set it up to have my comfortability.

This one runs CentOS 7 and for Linux based OS, Terminal is what you work with the most.

So, I started to configure it.

  • Change the background color and cursor of Terminal

The default color of Terminal is white background and block cursor. But I prefer black background and underline cursor because white background is so dazzlingly bright.

So, open up the Terminal, choose from the menu: Edit > Profile Preferences:

2017-05-10_110243

2017-05-10_110512

  • Change Bash PS1 colors for easy navigating and observing and show git branch information

Bash allows us to customize color and appearance of Terminal information, below is my own style. For more information for your own customization, refer this link.

And I also work quite much with Git so I need to visible branch information in Terminal.

# get current branch in git repo
 function parse_git_branch() {
 BRANCH=`git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\1/'`
 if [ ! "${BRANCH}" == "" ]
 then
 STAT=`parse_git_dirty`
 echo "[${BRANCH}${STAT}]"
 else
 echo ""
 fi
 }

# get current status of git repo
 function parse_git_dirty {
 status=`git status 2>&1 | tee`
 dirty=`echo -n "${status}" 2> /dev/null | grep "modified:" &> /dev/null; echo "$?"`
 untracked=`echo -n "${status}" 2> /dev/null | grep "Untracked files" &> /dev/null; echo "$?"`
 ahead=`echo -n "${status}" 2> /dev/null | grep "Your branch is ahead of" &> /dev/null; echo "$?"`
 newfile=`echo -n "${status}" 2> /dev/null | grep "new file:" &> /dev/null; echo "$?"`
 renamed=`echo -n "${status}" 2> /dev/null | grep "renamed:" &> /dev/null; echo "$?"`
 deleted=`echo -n "${status}" 2> /dev/null | grep "deleted:" &> /dev/null; echo "$?"`
 bits=''
 if [ "${renamed}" == "0" ]; then
 bits=">${bits}"
 fi
 if [ "${ahead}" == "0" ]; then
 bits="*${bits}"
 fi
 if [ "${newfile}" == "0" ]; then
 bits="+${bits}"
 fi
 if [ "${untracked}" == "0" ]; then
 bits="?${bits}"
 fi
 if [ "${deleted}" == "0" ]; then
 bits="x${bits}"
 fi
 if [ "${dirty}" == "0" ]; then
 bits="!${bits}"
 fi
 if [ ! "${bits}" == "" ]; then
 echo " ${bits}"
 else
 echo ""
 fi
 }

export PS1="\[\e[31;40m\]\u\[\e[m\]@\[\e[33;40m\]\H\[\e[m\]:\[\e[36;40m\]\w\[\e[m\]\[\e[32m\]\`parse_git_branch\`\[\e[m\]\n"

Add above to .bashrc (sudo gedit ~/.bashrc and paste above scripts to the end of the file) file and the result:

2017-05-10_111541

P/s: don’t forget to run source ~/.bashrc to make it takes effects.

  • Setup git autocompletion

It supports us to work faster and more accurate with git commands. So to make it happens, I use a guide from here:

Get the autocompletion script:

curl https://raw.githubusercontent.com/git/git/master/contrib/completion/git-completion.bash -o ~/.git-completion.bash

Add the below script to the end of the ~/.bashrc file (sudo gedit ~/.bashrc):

test -f ~/.git-completion.bash && . $_

Fire up changes:

source ~/.bashrc

Now the Terminal is ready to be worked on. Yay.

How to deploy your hubot to Heroku and why your bot is not working after deploying

After completing the development of my Hubot, I decided to deploy it to Heroku.

I followed the tutorial on Github’s Hubot guideline as here.

For, short:

$ cd [path-to-your-hubot-folder]
$ heroku login

# Setup git (if not yet).
$ git init
$ git add .
$ git commit -m "Initial commit"

# Create a new heroku application.
$ heroku create

# Push to heroku master.
$ git push heroku master

# Set Slack token for heroku (Can be found at
# https://[your-slack-channel].slack.com/home 
# and under your Hubot app config).
$ heroku config:set HUBOT_SLACK_TOKEN=[xoxb-YOUR-TOKEN]

# Set heroku keep alive url (To keep your bot active).
$ heroku config:set HUBOT_HEROKU_KEEPALIVE_URL=$(heroku apps:info -s  | grep web-url | cut -d= -f2)

But my bot is not working at all, it keeps sleeping on the new Heroku’s bed and not responded to any of my commands.

I checked the build log on Heroku dashboard, it shows a lot of node modules are lack.

I viewed the .git_ignore file and its content:

node_modules
.DS_Store*
.hubot_history

So entire node_modules is ignored during commit and pushing to Heroku. So I think that it’s the reason why my bot is not working after deploying.

I forced to add the folder with -f parameter in git add command (or you can just remove the line in .git_ignore folder), then commit and push again to Heroku.

And finally… the monster awakens.