Tuesday, January 11, 2011

Down the Foxhole - ActiveDen

Down the Foxhole - ActiveDen

Link to Envato Notes

SafeShell

Posted: 11 Jan 2011 03:30 PM PST

Need to call out to shell commands to process user-submitted files from your Rails app? You should be using our safe_shell gem.

The Problem

Let’s say a friendly user has uploaded a file called “avatar.jpg”, and you’re using ImageMagick to find out about it. In your app you do:

info = `identify #{filename}` 

That'll expand to:

info = `identify avatar.jpg` 

All good, right?

Now a malicious user comes along and uploads a file called ";rm -rf .". Now your command expands to:

info = `identify;rm -rf .` 

Uh oh. Because the backtick operator forks a shell, and the shell parses the command, this will happily do exactly what you don't want it to. Bye bye anything your in your Rails app that can be deleted.

So what's the answer?

How Not To Fix It

Well, you could try escaping the filename before you put it in, but the various methods of escaping available in Ruby each have their own problems. Besides, why escape the string, then fork a shell to immediately unescape it again?

You could use exec instead of backticks. That doesn't fork a shell, and lets you simply pass the arguments to the command you want to run as method arguments:

exec("identify", filename) 

Problem is, that doesn't return the output of the command, which is what you were after in the first place.

SafeShell To The Rescue

Enter SafeShell, riding a gleaming GitHub-shaped steed, and encased in a precious gem:

info = SafeShell.execute("identify", filename) 

This calls the given shell command directly, the same way exec does, but returns you the resulting output. No mess, no fuss.

It's got a few other little tricks up its sleave, but you can read about those on GitHub.


Meet the Staff: Josh Johnson

Posted: 10 Jan 2011 11:42 PM PST

Every week we'll introduce you to an Envato staff member or team. This week we meet Josh Johnson, editor of iPhone.AppStorm, where this article was originally published.

When working as an editor or writer online, it's easy to stay behind the scenes and remain something of an elusive shadow. AppStorm is no exception – we have a great team of editors and writers who work really hard to bring you a regular stream of top-notch articles across the sites.

Today, I thought it would be fun to publish an interview with your very own editor – Josh Johnson. He joined the AppStorm team just a few months ago, and is the driving force behind everything that goes on at iPhone.AppStorm.

I hope you enjoy getting to know your editor a little better, and feel free to leave a few questions for him in the comments!

How did you begin working for Envato/AppStorm?

I started writing for Mac.AppStorm in May, 2009 and quickly became addicted. David Appleyard and I became fast friends and have since begun working together on a number of other projects. The opportunity transformed my entire career and actually led to me leaving my full-time work as a graphic designer to focus on writing.

Before I knew it I was doing so much for AppStorm on a daily basis that the good people at Envato decided to let me take over one of the sites.

What part of your job do you enjoy the most? And which part really grinds your gears?

I'm a complete Apple nerd so my favorite part is absolutely trying out new apps and writing about what I think of them. It's amazing to me that I can actually make a living doing something that is so ridiculously fun.

My least favorite part of the job is probably sorting through the mounds of email I receive on a daily basis from various projects I'm working on. I love meeting and interacting with people but finding time to go through, respond to and act upon every message I receive can be exhausting.

Which iOS apps couldn't you live without?

I love productivity and organization apps so Evernote, Producteev and TeuxDeux are on the top of my list. Apps like these really have the power to change the way I work for the better and I'm always on the lookout for great finds in this category.

I'm also predictably addicted to social media so Facebook for iPhone and Twitter for iPad are definitely two of the apps that you'll see open most frequently on my devices.

Talk us through your computer setup – both hardware, and software!

Hardware

I do most of my work on a 2.16GHz Intel Core 2 Duo MacBook. I've had it for a few years and it's definitely starting to age – but it helps that I recently maxed out the RAM and upgraded to a 500GB 7200RPM hard drive.

On top of this I own a second generation 20GB iPod that amazingly still works (it weighs a ton and I'm certain it's bullet proof), a first generation 16 GB iPod Touch, a 64GB iPad, an iPhone 4, and a Time Capsule – just in case any of it ever dies!

Software

For writing, I prefer the simplicity of TextEdit and WriteRoom. I find that when I fill my screen with one of these two apps I'm much more productive than working in a full-blown word processor with tons of formatting options that don't really matter (especially when I'll just be copying and pasting plain text into WordPress). I'm not the biggest Microsoft Office fan anyway so it's a good day when I can avoid that suite altogether.

Since, I'm still very much involved in design you'll always see Adobe Photoshop and Illustrator (CS5 at the moment) open on my Mac as well. Other Mac apps that I absolutely couldn't live without include the following:

  • Dropbox: Unbeatable simplicity for online backup and file sharing.
  • TextExpander: This app saves me so much time on a daily basis that I can't imagine working without it.
  • ClipMenu: The best free clipboard manager on the planet.
  • LittleSnapper: Being a full-time blogger means taking hundreds of screenshots every month. LittleSnapper is a lifesaver for saving and organizing everything that I snap.
  • Evernote: My entire life is in Evernote and I'm convinced that if it ever goes away I'll be completely lost.
  • Tweetie: Its lack of development is frustrating but I've tried every Mac Twitter app in existence and nothing beats Tweetie in the features that I need the most: multiple accounts, user profiles, saved searches, etc.
  • Safari: I go through phases where I switch to Firefox, Chrome, or even Camino for months on end but I always end up right back with Safari. Now that it finally has a decent extension system I don't think I'll ever switch.
  • Espresso: It's simply the best app around for building basic websites. I wouldn't dream of using anything else.
  • Fluid: I use fluid to turn all my favorite web apps into Mac Apps: Producteev, TeuxDeux, I even have an awesome Facebook menu bar app that's really just a Fluid instance of the Facebook mobile site.

Which post on iPhone.AppStorm are you most proud of having published?

15+ Incredibly Useful iPhone Tips and Tricks

That article was a ton of fun to research and write. I learned a lot both from the tricks that I found and those that the commenters suggested.

One of my favorite things to do with any system is dig and and find all the little features that most people don't know about. I wrote a similar article for Mac.AppStorm.

What do you get up to when you're not editing our awesome articles?

I'm also the editor of Design Shack, and occasionally write for a few other design blogs such as Six Revisions and Smashing Magazine.

In recent years I've taken up photography and am really loving it as yet another creative outlet. You can see some of my work at SmugMug.

When I'm not working I enjoy spending time with my amazing wife Kelley and playing guitar weekly at our church.

What's the best thing that you've watched, listened to, and read this week?

The best thing I watch each week is usually whatever web development voodoo Jeffery Way is cooking up for the weekly Quick Tip at Nettuts+ (the latest is on JavaScript Hoisting). I've learned a ton from watching his screencasts and am continually impressed with his knowledge of JavaScript.

The best thing I've listened to this week is almost the only thing: The Beatles. In honor of Lennon's 70th birthday I've had my several Beatles albums repeating in iTunes. You can't beat the originals but I do quite enjoy the "Across the Universe" movie soundtrack as well.

In addition to the countless design blog posts and news articles I read on a weekly basis, I really enjoy novels, especially the classics. I'm currently reading "The Count of Monte Cristo" and "Alice's Adventures in Wonderland" in addition to a couple of newer novels: Grisham's "The Appeal" and Crichton's "Next".

As you can tell, I'm all over the place with with genres but I tend to favor books with a fantastical setting such "The Lord of the Rings" or even "1984″.

What features would your ultimate mobile Apple gadget have?

My ultimate mobile gadget would be very close to the iPad. Ideal improvements would be dual processors, a couple of cameras with video-conferencing, wireless syncing, the ability to run full-fledged Mac apps, better multi-tasking, integrated text snippets to save on typing and maybe a tiny built-in projector with the ability to interact with the projection.

It would also definitely have wireless charging, and I don't mean that lame charger you see on the Palm Pre. I mean full-on over the air charging via WiTricity or something similar.

Having your devices charge just by virtue of being in the same room as a charger sounds like something from science fiction, but it's more than possible using the basic principles discovered by Tesla ages ago and I think it's about time we started putting this fantastic technology into practice in cell phones, laptops, tablets and more.

If you could ask your readers one question, what would it be?

What could we do to make AppStorm a site that you visit every day or at least every week?

Thanks, Josh!

Thanks to Josh for taking the time to shed a little light on how he came to edit iPhone.AppStorm, and what his routine looks like!

If you have a question for him, or would like to answer his question above, we'd love to chat in the comments. Now's your chance to quiz your editor on anything and everything related to AppStorm and iPhone apps (no pressure, Josh!)


Rebasing Merge Commits in Git

Posted: 10 Jan 2011 07:29 PM PST

I’m one of the devs here at Envato, and this is my first post to the Notes blog. Having found myself in a role I would cautiously describe as ‘resident Git expert’, it’s only fitting that my first post would be about a fairly technical aspect of working with Git in a team environment.

The TL;DR version is this: When rebasing, always use the -p flag,

First, though, a small diversion – why rebase is part of my normal git workflow.

Why I starting using pull –rebase

Using git pull --rebase is becoming more popular to avoid unnecessary merge commits when fetching the latest code from master. There are a few blog posts on the matter, such as [1] [2]

I’ll give a brief summary of why this convinced me. For me, it boils down to two simple cases:

1. You haven’t made any changes to your branch

In this case, pull and pull --rebase will simply fast-forward. No problems.

2. You’ve got one or two small changes you forgot to push

In this case, the default pull will actually merge the remote changes into your branch, making a merge commit. This is bad for a couple of reasons, messiness is one, but I actually consider the problems it causes for git bisect more compelling (I must remember to write about that one day).

With git pull --rebase, you simply replay those commits on top of the new head. Now, if you push, you have linear history, rather than a divergence/merge. I think this is a better result. Usually, I follow the ‘always work on a branch’ and ‘merges are meaningful and good’ practice (partly inspired by [3]), but there’s no semantic difference between master and origin/master, so linear history makes sense.

So in general, git pull --rebase is better than a git pull. To make it the default, see [4].

There is one major problem with it, though – merge commits.

Rebasing deletes merge commits

This is best explained with an example

First, one that doesn’t fail

Given this simple repo:

Initial state

[master] git pull --rebase First, rewinding head to replay your work on top of it... Applying: little fix Applying: forgot to push this 

After pull

[master] git push Counting objects: 6, done. Delta compression using up to 8 threads. Compressing objects: 100% (4/4), done. Writing objects: 100% (5/5), 526 bytes, done. Total 5 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (5/5), done. To /Users/glen/envato/demo-origin    f9c3cb8..e4a2e92  master -> master 

After push

Linear history, just as we wanted!

All aboard the failboat

Say you’ve been working on your little feature for a while, like this:

Initial state

Then you merge to master (using --no-ff, of course [3])

[master] git merge --no-ff feature Merge made by recursive.  b |    3 +++  1 files changed, 3 insertions(+), 0 deletions(-)  create mode 100644 b 

Merged

Then you go to push, but somebody got in there first (origin/master has moved on)

[master] git push To /Users/glen/envato/demo-origin  ! [rejected]        master -> master (non-fast-forward) error: failed to push some refs to '/Users/glen/envato/demo-origin' To prevent you from losing history, non-fast-forward updates were rejected Merge the remote changes (e.g. 'git pull') before pushing again.  See the 'Note about fast-forwards' section of 'git push --help' for details. 

Of course, trying to push hasn’t updated our reference to origin/master, we need to git fetch to see the full picture

[master] git fetch remote: Counting objects: 5, done. remote: Total 3 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (3/3), done. From /Users/glen/envato/demo-origin    49ab1cf..9f3e34d  master     -> origin/master 

Fetched
Ah, yes. Someone has pushed a commit ‘sneaky extra commit’ before we were able to push our commit (merging in of branch feature). So, we would normally just git pull --rebase to get ready to push, but if we do that, the merge commit gets deleted!

[master] git pull --rebase First, rewinding head to replay your work on top of it... Applying: my work Applying: my work Applying: my work 

Doom

Our merge commit has disappeared!

This is bad for a whole lot of reasons. For one, the feature commits are actually duplicated, when really I only wanted to rebase the merge. If you later merge the feature branch in again, both commits will be in the history of master. And origin/feature, which supposed to be finished and in master, is left dangling. Unlike the awesome history that you get from following a good branching/merging model, you’ve actually got misleading history.

For example, if someone looks at the branches on origin, it’ll appear that origin/feature hasn’t been merged into master, even though it has! Which can cause all kinds of problems if that person then does a deploy. It’s just bad news all round.

Worst of all, you did everything ‘right’. You used merge --no-ff and git pull --rebase. Sad face.

In case it’s not obvious, this is what we wanted to happen:

Ideal outcome

You can recover from this situation (if you discover it before you push) by resetting and redoing the merge:

[master] git reset --hard origin/master HEAD is now at 9f3e34d sneaky extra commit [master] git merge --no-ff feature Merge made by recursive.  b |    3 +++  1 files changed, 3 insertions(+), 0 deletions(-)  create mode 100644 b 

The solution!

In the manpage for git-rebase

-p --preserve-merges Instead of ignoring merges, try to recreate them.  This uses the --interactive machinery internally, but combining it with the --interactive option explicitly is generally not a good idea unless you know what you are doing (see BUGS below). 

Or, to put it another way:

AWESOME

So, instead of using git pull --rebase, use a git fetch origin and git rebase -p origin/master:

[master] git fetch remote: Counting objects: 5, done. remote: Total 3 (delta 0), reused 0 (delta 0) Unpacking objects: 100% (3/3), done. From /Users/glen/envato/demo-origin    49ab1cf..9f3e34d  master     -> origin/master 

Fetched

[master] git rebase -p origin/master Successfully rebased and updated refs/heads/master. 

Ideal outcome
Win!

Downsides

Git pull is dead

The -p flag doesn’t apply to git pull --rebase, so you have to start explicitly fetching and rebasing. To be honest, I think this is more an upside. Fetching explicitly is good, since it refreshes your entire copy of the remote, and lists what branches have moved on (handy on a fast-moving project). But for those used to a single-step pull, this is slightly more work.

ORIG_HEAD is no longer preserved

ORIG_HEAD, once you get used to using it, is really handy to undo a destructive operation. Sadly, git rebase -p sets ORIG_HEAD for each commit being rebased, so you can’t use it to quickly return to the start of a rebase, something I ran in to working on this post.

Branch tracking is not used

Unlike git pull --rebase, which will fetch changes from the branch your current branch is tracking, git rebase -p doesn’t have a sensible default to work from. You have to give it a branch to rebase onto. With a good alias, however, that can be made painless.

Aliases

So, how about some aliases to make this all idiot-proof? I’ve decided to call mine gup (I’ve taken to calling it gee-up), and I’ve got it in a gist for bash, fish and zsh here. I’ve also included my gpthis alias for pushing without branch tracking.

gup will do a fetch of origin and rebase -p of the branch on origin with the same name as the current branch. For 99% of cases, this is exactly what I want.

Conclusion

This morning, I had no idea about the --preserve-merges flag on rebase, and was about ready to cry foul on using rebase at all, considering how bad this problem can get on a big project. But, as with everything Git, once you understand it a bit better, there’s usually a more complex way that sucks a whole lot less. Which is why aliases like gup are really handy – you can keep changing what your aliases mean without having to learn a new habit.

I’d welcome comments and suggestions, you can either reply here or hit me up on twitter – @glenmaddern


No comments:

Post a Comment