Managing local changes using “git stash” tool

(Disclaimer: I am quite new to git so I may end-up making a mistake here!)

Use-case:

You are using a third-party software codes via it’s git repo. For some reason, say¬†that software’s developer does not have support for plugin or customization or the support for changes is limited. Somehow you need to make changes to codes but since changes are useful to only you, they reject your pull requests.

You still want to keep using this software and want to preserve your changes across future updates! Here comes git’s stash tool to your rescue! ūüėČ

Workflow:

Our workflow will have mainly 3 steps:

  1. Save your local changes (git stash save)
  2. Upgrade local git repo (git fetch & git checkout [...])
  3. Re-apply (merge) local changes to new codebase (git stash apply)

#1. Saving local changes

If you have local-changes and if you decide to update your codes (git pull/git checkout to latest tag), its quite likely that git will give you a hint to either discard or stash your changes.

Since you do not want to discard them, lets stash them using simply

git stash

Or

git stash save "some useful name/comment"

When you run a command like above, all your local changes gets saves to a stack. You don’t need to worry about where this stack resides, or other internal details. You can imagine that your local modifications are stored in a blackhole! ūüėČ

#2. Upgrade local codebase

This involve mostly fetching remote changes and switching to latest tag (or branch if they don’t like tagging).

Since we are dealing with versions, I feel using git pull is not necessary. git pull does git fetch which is followed by git merger. As we are on a tag (which is not supposed to change by convention) there must be nothing to merge on it from remote.

I generally use following set of commands for upgradation. YMMV.

git fetch
git tag -l
git checkout <latest-tag>

#3. Reapplying local changes to latest codes

Since that developer did not incorporated your changes, you will find latest codes do not have your customizations.

You can reapply them using

git stash apply

Or specifying stash index (in case you did stash save many times)

git stash apply [email protected]{2}

To get exact index number, in case of multiple stash-saves, please use following command

git stash list

My changes got reapplied successfully without any issue. ūüėČ

I guess depending on changes, we may need to do some adjustments as we do when doing merge.

Bonus: Undoing Stash Apply!

Just in case you applies wrong stash, or apply it on a wrong tag/branch, you will need a way to undo it.

If you are maintaing only one stash, then simply run…

git stash show -p | git apply -R

OR specify stash index

git stash show -p [email protected]{2} | git apply -R

If you know any better way, feel free to share with us! ūüôā

(image credit: created in-house by our design Yogesh Daphane. Idea was mine!)

Using grep to removing comments, newlines/whitespace from config files

As configuration files are often full with comments and blank lines reading them inside shell become tedious! Sometimes you just want to quickly recap active configuration option only.

Mostly, comments starts with¬†“#“(hash) or “;” (semicolon).

Below is a command I often use while debugging various configuration files.

For files having comment starting with “;”¬†e.g. php.ini

egrep -v "^([[:space:]]?$|;)" /path/to/file

For files having comment starting with “#” e.g. my.cnf

egrep -v "^([[:space:]]?$|#)" /path/to/file

For lazy bumps…¬†Following can take care of both of above!¬†

egrep -v "^([[:space:]]?$|;|#)" /path/to/file

You may create a local alias for above command! ūüėČ

Just in case you want to save output in a file… just add " > newfilename" to above commands. Passing original filename may not work (and it will be risky too!)

Fix File Permission on Linux/Mac Server

Over the time, file permissions get messy on a linux/mac server. It really annoys (me) to see static files like images have 0755 i.e. executable permission set for them.

You can use following commands to fix permissions:

find . -type d -print0 | xargs -0 chmod 0775
find . -type f -print0 | xargs -0 chmod 0664

I found many scripts and tools to do this job. IMHO, above command does what we really need!

For WordPress

Ideally, on a wordpress site, no files under “wp-content/uploads”¬†directory¬†should be executable.¬†In case of WordPress multisite, it will be¬†“wp-content/blogs.dir” directory.

Its better to fix file-permissions periodically on your WordPress setup.

Ubuntu Server + CrashPlan + Backup + Remote Management using Mac

We already use rsanpshot for backups. But its always feel safe to have more than one kind of backup.

For other type of backup, we use Crashplan. They support all major platforms, provides many options and have their service decently priced. It will take a separate post to list down all things I like about Crashplan so lets get back to work here!

What I wanted to do:

  1. Run Crashplan on Ubuntu Headless Server.
  2. Manage it using Crashplan desktop software on my Mac.
I guess you can start remote X session or something like that on a Ubuntu server but that will be overkill for server. Specially when you can achieve above tasks in just 2 minutes using following commands.

Preparation

Crashplan on Ubuntu will need Java Runtime (JRE). Crashplan can install it itself but I prefer to have more control over which JRE I end up putting on our server. They have almost 5 version of JAVA for Ubuntu!

Anyway, just run following command as root user or with sudo access:

apt-get install openjdk-7-jre-headless

Install Crashplan

Next, we will install Crashplan directly.

Go to Crashplan site, select Linux version and copy-download link to grab latest Crashplan version. Crashplan 3.2.1 is latest as of July 2012.

Run following commands to download, uncompress and install CrashPlan or CrashPlan+

wget http://download.crashplan.com/installs/linux/install/CrashPlan/CrashPlan_3.2.1_Linux.tgz
tar -zxvf CrashPlan_3.2.1_Linux.tgz
cd CrashPlan-install
./install.sh

If you are using CrashPlan PRO, use following…

wget http://download.crashplan.com/installs/linux/install/CrashPlanPRO/CrashPlanPRO_3.5.3_Linux.tgz
tar -zxvf CrashPlanPRO_3.5.3_Linux.tgz
cd CrashPlanPRO-install
./install.sh

Installer will ask you for many questions. Just keep hitting enter keys. Defaults options are all good to go ahead!

At this point Crashplan is running on your Ubuntu Server but it’s not backing anything yet! We will use Crashplan’s Desktop edition for Mac to start backup.

Troubleshooting

If your installation ends with warning:

Your Linux system is currently configured to watch 8192 files in real time.
We recommend using a larger value; see the CrashPlan support site for details

Then you need to tweak sysctl file:

/etc/sysctl.d/custom.conf

Add…

fs.inotify.max_user_watches=1048576

Save changes…

sysctl -p

Remote Management UI (using Mac)

Go to Crashplan’s site. This time download Crashplan for Mac &¬†Install it.

If you have started Crashplan after installation, quit it.

Open terminal to edit Crashplan config file…

vim /Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties

You can use any text-editor in case you don’e like Vim.

Find a line like below:

#servicePort=4243

Change it to:

servicePort=4200

Note that we have changed port number from 4243 to 4200 (in case you thought we just uncomment it!).

Save file. Exit vim.

Create SSH Tunnel from your mac to remote Ubuntu server using a command-like below:

ssh -L 4200:localhost:4243¬†[email protected]

Replace username and hostname with your a real username on your hostname. You can put your server IP address also.

After you run above command, you will be logged into a shell on your server. Keep it running as its connecting your desktop crashplan to your server crashplan!

Finally, you can start your Crashplan on Mac and configure backup for your Ubuntu Server!

In case you are already using Crashplan to manage your Mac’s local backup:

  • You will need to exit from shell (to break connection)
  • comment-out “servicePort” line in Crashplan config
  • Restrat Crashplan so it will work with Mac’s local config

There is no easy-way to switch between local & remote machines. (Atleast at the time of writing this…)

Solution for “Locale Error – perl: warning: Setting locale failed.” [Ubuntu]

We are maintaining old version of Ubuntu servers (10.04 LTS to be specific)

Many times we encounter following errors on terminal:

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LC_CTYPE = "UTF-8",
    LANG = "en_GB.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

Client-Side Solution

This is good option if you manage many servers. Rather than changing every server, pass requested environment.

Open ~/.bash_profile

Add following line:

export LC_ALL=C

Start a new shell or just reload environment using source ~/.bash_profile

Server-Side Solutions

Solution #0

This is new but working nicely.

update-locale LC_ALL=$LANG

Solution #1

export LANGUAGE=en_IN.UTF-8
export LANG=en_IN.UTF-8
export LC_ALL=en_IN.UTF-8
locale-gen en_IN.UTF-8

If for some reason, above failed to work and throws following fresh error:

warning: setlocale: LC_ALL: cannot change locale (en_IN.UTF-8)

Try following alternative solution.

Solution #2

Edit locale file:

vim /etc/default/locale

Add following line:

LC_ALL="en_IN.UTF-8"

Generate locale again:

locale-gen en_IN.UTF-8

Solution #3

Open following file

vim /var/lib/locales/supported.d/local

Make sure it has

en_IN.UTF-8 UTF-8

Then run…

dpkg-reconfigure locales

Test

Run following command:

locale

If you see any error, then that means your issue is not fixed!

Try again. There are some more alternate solutions that you can find on Google.

Above seemed to work for me always. ūüôā

MaxCDN – Updating IP Address when moving servers

If you use MaxCDN to speed-up your WordPress blog/site, then you may face one issue during server migration. The issue is –¬†IP-address of new server will not get updated in MaxCDN automatically. You need to do this manually like below:

Steps to Update IP Address on MaxCDN:

  1. Login to MaxCDN account. Go to “Manage Zones”.
  2. Go to “Pull Zone” which you like to update.
  3. Then go to “Settings” tab.
  4. You will see a section titled “Origin Information”. ¬†That will contain IP address of your old server. Click the “Edit” button in that section. (see screenshot)¬†
  5. Then enter new IP-address in “Origin IP” field and click the “Update” button. ¬†(see screenshot)¬†

Why don’t MaxCDN update IP address manually?

Its common to perform some testing on a test server, and then use another server, a different one, for production site. So IP-address change is not something very uncommon.

MaxCDN uses IP address to serve content faster. But rather than automatically updating their cache, say daily, they keep it forever! While I like the manual option to “Update” IP address is settings area, which gives advance users some control, it would be better if MaxCDN combine this manual option with automated cache-update.

Smart Auto-Update:¬†A smart solution would be to update IP-Address when “Origin Server” starts returning HTTP 404-error. When there will be too many 404-errors, it may mean the server has been moved.¬†Of course, CDN misconfiguration or origin server-downtime can also result in 404-error but “Smart” programs supposed too handle such cases as well!

Recommended Reading: How to Setup MaxCDN on WordPress using W3 Total Cache Plugin

Remove “eval(base64_decode” using linux commands from all php files across multiple WordPress

Yesterday, almost all installations on our test server had been infected by infamous “<?php¬†eval(base64_decode(…)) ?>” code injection.

We have more than 600 demo sites on our test server and cleaning them using any WordPress plugin out there was simply out of the question! Can you imagine logging into each WordPress,¬†installing¬†plugin, then scanning/cleaning up WordPress… for 600+ WordPress sites?

Below is combination of Linux commands we used.  Assuming you have logged into a Linux Shell and already have BACKUP of all files (including infected files) lets move ahead!

Command to list all infected files:

grep -lr --include=*.php "eval(base64_decode" /path/to/webroot

This is not necessary but its better to check some files manually to confirm if they have malicious code we are looking for. Also we can use this command after running cleanup command to crosscheck if cleanup is really successful.

Command to remove malicious code:

If above command gives you correct output, execute following command to perform actual cleaning:

grep -lr --include=*.php "eval(base64_decode" /path/to/webroot | xargs sed -i.bak 's/<?php eval(base64_decode[^;]*;/<?php\n/g'

Executing above will remove eval(*) codes.  Above command will also generate a backup version of files it will modify. For example, if it removes code from index.php, you will find a new file index.php.bak in same directory with original content of index.php

Now after running above command, you still find some more infected files, then you need to adjust search and replace parameters in for “sed” part. ¬† ¬†You may also use following command for a “liberal” cleaning at the risk of breaking something. (in case you really break something, like I did, you can jump to “Troubleshooting” section below!)

grep -lr --include=*.php "eval(base64_decode" /path/to/webroot | xargs sed -i.bak '/eval(base64_decode*/d'

Trying to avoid re-appearance of this code injection

Its really though to cover every possible way to protect yourself from such attach in this post.

If you remember, WordPress community faced this kind of issue because of WP-PhpMyAdmin plugin sometime back. In our case, we found some old WordPress demo sites were having that plugin installed.

To remove WP-PhpMyAdmin plugin form all WordPress sites on your server, execute following command:

find /path/to/webroot -name "wp-phpmyadmin" -type d | xargs rm -rf

Above is all we did to get rid of eval(base64_decode(*)) codes from all files on our test server. If this happens again on our server, I will update this post with added info.

Troubleshooting:

Just in case you end up in a mess, below are some useful commands.

Missing <?php tag in the beginning:

To add “<?php: tag in the¬†beginning¬†of index.php files, in case if you remove it¬†accidentally use following command:

find /var/www/ -name "index.php" | grep "/htdocs/index.php" | xargs grep -L "<?php" | xargs sed -i "1s/^/<?php \n/"

Don’t worry. If you already have a “<?php ” tag in the beginning, it won’t be added again.

Extra Newlines at the top!

If you find after cleanup, extra newlines at the top of your code, then use following command to remove trailing newlines. Extra newlines creates problem for blog feeds.

find . -name '*.php' -exec sed -i -e :a -e '/^\n*$/{$d;N;ba' -e '}' '{}' \;

I hope you will find this stuff useful.

Download “Devils’ Workshop” Blogger Template

Mukund from NewBloggingTipz.com released a free Blogger.com template which replicate our current Devils’ Workshop theme nicely. ūüôā

You can see live demo here.  You can download it from here.

If you decide to use it on your Blogger.com blog, please read follow-up articles on customizations as well.

As Mukund developed this theme on his own, contact him only for support. ūüėČ

WordPress Theme

I saw some readers requesting WordPress version of DW theme as well. We will surely release current DW WordPress Theme for community once its get ported on our recently launched rtPanel wordpress theme framework.

Thanks Surun for the tip. ūüôā

Link:¬†Devils’ Workshop Blogger.com Template

 

 

 

Important – Major Changes to rtBlogs Network

Hi All,

You might have noticed a major downtime today for Devils Workshop as well as other blogs in rtBlogs network.

We were pushing a major update where we merged all blogs in rtBlogs WordPress multisite inside¬†rtCamp’s WordPress multisite. The goal was to centralize our user’s login info so that they can easily contribute to our blog network, as well as use support forums at rtCamp’s product portals easily.

In the long run, you will be using a single account to:

Important Notes:

Our current move many have some side-effects. So please check few things:

  • If you are using our Google AdSense Revenue Sharing program, please check your posts to see if they are displaying your ads.
  • Also check if your posts are attributed to you only
  • If you are not able to log in, try resetting your password with email-address you used at the time of¬†registration.
  • If you see your user-name changes, please do not panic. We are expecting few usernames to be changed as we merged two user-bases.
We have triple-checked many things, but still in a big community like this, things may go wrong. Whatever issue you may encounter, do NOT panic!
You can use comment form below or our contact-form to reach us. Please give as many details as possible while contacting for technical support (related to above issues).

While today’s changes will most likely go unnoticed for a while, we hope to surprise you with our next round of changes. The good thing about next round is, there will be no downtime! ūüėČ