Fun With Python and Monkey Patching

Monkey patching is about replacing attributes of a Python thing with other attributes. Let's use the word "thing" very loosely and have some fun.

Messing With Self

First, the boring case where classes are classes and instances are instances.

	class MyClass(object):
	def __init__(self, msg):
	self.msg = msg
	def say_stuff(self):
	print("Say stuff message is: {}\n\n".format(self.msg))
	# normal call and instantiation
	ob1 = MyClass("Hello there, first example.")
	ob1.say_stuff()

view raw conventional_class_instance.py hosted with ❤ by GitHub

Nothing new there. But what is with self? What's so special about it? Let's rebel. Everywhere self is shall be replaced with the word her.

	class MyClass(object):
	def __init__(her, msg):
	her.msg = msg
	def say_stuff(her):
	print("Say stuff message is: {}\n\n".format(her.msg))
	# normal call and instantiation
	ob1 = MyClass("Hello there, This is me.")
	ob1.say_stuff()

view raw her_self.py hosted with ❤ by GitHub

It turns out self is not that special after all. It's just a convention for distinguishing the instance of a class from the class itself. Strictly speaking, self could be this or alice or bob or the gender pronoun of your choice. Politically correct Python for the win!

But seriously, did you ever stop to think about how strange it is when it comes to self and how calls method calls actually work?

ob1.say_stuff()

The ob1 in this case looks like it's the self that say_stuff refers to. Let's twist that call around a little.

	say_stuff = MyClass.say_stuff
	say_stuff(ob1)

view raw rearranged_ob_instance.py hosted with ❤ by GitHub

Okay, so there's that. Now, be sure of this. The self term absolutely refers to an instance of the class it's being used in ... right? You have to wonder sometimes.

	class MyClass(object):
	def say_stuff(self):
	print("Say stuff message is: {}\n\n".format(self.msg))
	class OtherClass(object):
	def __init__(self, msg):
	self.msg = msg
	other_ob = OtherClass("Hello from the other_ob instance")
	MyClass.say_stuff(other_ob)

view raw Other_Class_Instance.py hosted with ❤ by GitHub

So much for that idea. At any rate, it's safe to say we've abused self enough for now. 

Time For Some Actual Monkey Patching

Here, we play a game about filling in the missing pieces. What if we had a starting class without an __init__.  To make it more interesting, let's NOT set up another class through which to provide instance variables. Something like this.

	class MyClass(object):
	def say_stuff(self):
	print("Say stuff message is: {}\n\n".format(self.msg))

view raw missing_msg_instance.py hosted with ❤ by GitHub

And who would have thunk. We can actually make this work in spite of the fact that the instance was born without a msg. Time for some setattr() surgery.

	class MyClass(object):
	def say_stuff(self):
	print("Say stuff message is: {}\n\n".format(self.msg))
	ob = MyClass()
	setattr(ob, "msg", "Here is a message from a setattr")
	ob.say_stuff()

view raw setattr_fixed_class.py hosted with ❤ by GitHub

Oh heck, let's go gangbusters and just monkey patch a bare naked class together. Here's what that ends up looking like. Heck, we'll do up the instance too. Here it goes.

	class MyClass(object):
	pass
	ob = MyClass()
	def say_stuff(self):
	print("The say stuff message here is: {}\n\n".format(self.msg))
	setattr(MyClass, "say_stuff", say_stuff)
	setattr(ob, "msg", "Patched in message for patched in class")
	ob.say_stuff()

view raw setattr_class_and_instance.py hosted with ❤ by GitHub

What else to play with? Ooo, I know!

Direct Assignments And Dictionaries

Python classes seem mutable enough. Maybe you can even directly assign to one method to take the place of the original. 

	class MyClass(object):
	def __init__(self, msg):
	self.msg = msg
	def say_stuff(self):
	print("Standard say stuff message: {}\n\n".format(self.msg))
	def say_other_stuff(self):
	print("New say_stuff says this: {}\n\n".format(self.msg))
	MyClass.say_stuff = say_other_stuff
	ob = MyClass("This is my message here")
	ob.say_stuff()

view raw direct_assignment_method_replacement.py hosted with ❤ by GitHub

Awesome! That opens up quite a few possibilities right there.

Okay, one last thought. Python classes and instances have these things called dictionaries that dwell beneath the surface. Dictionaries represent methods and attributes.

Now, here's an idea. say_stuff isn't a member of the instance dictionary. It's a member of the class that the instance is based on. If there no say_stuff in the instance? No problem. Just look it up in the dictionary of the class instead.

Sooooo, what if we exploited the instance dictionary to subvert that expectation.

	class MyClass(object):
	def __init__(self, msg):
	self.msg = msg
	def say_stuff(self):
	print("Standard say stuff message: {}\n\n".format(self.msg))
	ob = MyClass("Here is the original message")
	def say_stuff():
	m = "Thought I'd say something else, didn't you?"
	print(m)
	#setattr(ob, "say_stuff", say_stuff)
	ob.__dict__["say_stuff"] = say_stuff
	ob.say_stuff()

view raw instance_overshadows_class_method.py hosted with ❤ by GitHub

And yes, that setattr trick that's commented out there works too. And there are other things to say and do with these underlying dictionaries too. However, exploring much further requires  diving into the mysterious world of metaclasses. Let's not go there.... unless you really want to. ;-)

Well That Was Fun!

This is definitely not something you'd likely want to do with any REAL project unless you absolutely had to. Overuse of monkey patching can make things messy and confusing. At any rate, I hope you enjoyed this read. Have a great week.

Drag and Drop Files with HTML5 and Flask

Doing drag and drop file uploads has interesting uses. Uploading resumes, pictures, and so on are all scenarios where this is handy. So here's a look  at things I think about when it comes to drag and drop file uploads.

The Layout Of The Drop Zone

Since files need to go somewhere, we start by setting up a drop area for them to land in. This just requires a little HTML and CSS. The aim here is a simple box with a border. Code like this should do the trick.

	<style>
	.droparea {
	width: 200px;
	height: 200px;
	border-style: solid;
	border-width: 3px;
	border-color: red;
	float: left;
	line-height: 200px;
	text-align: center;
	}
	</style>
	<div class="container">
	<div class="droparea">Drop Here</div>
	<div class="tablearea" id="tablearea"></div>
	</div>

view raw dropzone.html hosted with ❤ by GitHub

The result looks like this.

And that's all on the look of it. As for DOING something with it, that's another story.

Getting Javascript/JQuery to Handle Drag and Drop

When dragging and dropping into that red square, the browser needs to know how to react. By default, it has its own ideas. That div element will tell you to buzz off if we try dragging stuff over it with the mouse. So we'll throw in a preventDefault invocation to say to the browser "Relax, I got this."

Then there are the drops which are their own kettle of fish. Often times, browsers seem to like popping open a new tab to display dropped content. We don't want that so another preventDefault heads that problem off at the pass.

Now, as for the file, we have to dig into the event to this thing called a dataTransfer. To me, dataTransfer plays the role of a handbag to store things in when taking dragged stuff from point A to point B. I don't know if that's the official definition but, for our purposes, it works.

	var dragHandler = function(evt){
	evt.preventDefault();
	};
	var dropHandler = function(evt){
	evt.preventDefault();
	var files = evt.originalEvent.dataTransfer.files;
	console.log(files[0]);
	};
	var dropHandlerSet = {
	dragover: dragHandler,
	drop: dropHandler
	};
	$(".droparea").on(dropHandlerSet);

view raw drag_drop_files_1.js hosted with ❤ by GitHub

For the moment, we just fish out the file that got stored there when we dragged it in. Then, we just display the name in the Chrome console. Hey look, a file!

And now I'm bored of Javascript. Let's switch to server side and talk Python. We'll deal with the rest of the client-side of things later.

Using Python And Flask To Give Files Somewhere To Go

Time to escape the browser and hang out in Pycharm and iPython for awhile. The mad plan is to knock out server code that takes in just any old file that wants to come live there.

	app.config["UPLOAD_FOLDER"] = "uploads"
	@app.route("/sendfile", methods=["POST"])
	def send_file():
	fileob = request.files["file2upload"]
	filename = secure_filename(fileob.filename)
	save_path = "{}/{}".format(app.config["UPLOAD_FOLDER"], filename)
	fileob.save(save_path)
	return "successful_upload"

view raw upload_file_1.py hosted with ❤ by GitHub

The code does what it says. We grab the file out of the requests "files" dictionary. Then we put the file name through the Flask "secure_filename" to help prevent users messing with the system via sneaky file names. Save it to the folder and we're done.

Back to Javascript yet? Nah. We can get by fine using requests and ipython to test this code. Banging out something like this should do the trick.

In [11]: from requests import post
In [12]: url = "http://localhost:5000/sendfile"
In [13]: res = post(url, files={"file2upload":open("junkloop.sh", "rt")}
   ....: )
In [14]: res
Out[14]:
In [15]: res.text
Out[15]: 'successful_upload'

Okay, that was easy enough. Check the uploads folder .... it's there. Good. What's next?

Back To Javascript To Upload This Stuff

We can't avoid the client forever so let's get back on that. The thing to do next is to put together a request in a way that does a real upload. To do that, we have to take into account funny quirks as to how browsers like treating file uploads.

One thing to take into account here. Browsers seem to like to turning files into strings that conform to some content type or another. We don't want to mess with content types here. The file should be uploaded just as it is. So what do we do?

Fortunately, jQuery give you a couple of options such as processData and contentType to get around that issue. After making proper modifications, file dropping logic that looks like this.

	var dropHandler = function(evt){
	evt.preventDefault();
	var files = evt.originalEvent.dataTransfer.files;
	var formData = new FormData();
	formData.append("file2upload", files[0]);
	var req = {
	url: "/sendfile",
	method: "post",
	processData: false,
	contentType: false,
	data: formData
	};
	var promise = $.ajax(req);
	};

view raw javascript_file_upload.js hosted with ❤ by GitHub

Now, just double check that uploads folder and say "Hooray, drag and drop uploading works!"

Bonus Round - Listing Uploaded Files

We might as well list the files that are being uploaded. Triggering a fresh list of current files is easy enough. Just attach a request for a fresh list of files as a success response to the upload request. Might as well make the most of those promises, right?  Adding something to the end of dropHandler would be a good start.

        var promise = $.ajax(req);
        promise.then(fileUploadSuccess);

Beyond which you need to implement that function so it actually takes care of getting that fresh list. Easy enough to implement. Standard handlebars templating. Click here and here if curious.

Okay, so then what? That /filenames endpoint in the Flask code. We really should take care of that. But first, a quick tweak to send_file. A line like this after saving the file would be nice.

    # open and close to update the access time.
    with open(save_path, "r") as f:
        pass

The point of this is that I want to see the files in the order that they are uploaded. Doing a quick open/close changes the access time. Now we have files with a chronological upload history.  Coolness!

Oh yes, that /filenames thing. Here's that.

	@app.route("/filenames", methods=["GET"])
	def get_filenames():
	filenames = os.listdir("uploads/")
	modify_time_sort = lambda f: os.stat("uploads/{}".format(f)).st_atime
	filenames = sorted(filenames, key=modify_time_sort)
	return_dict = dict(filenames=filenames)
	return jsonify(return_dict)

view raw flask_filename_list.py hosted with ❤ by GitHub

Well, that was kind of weird and hacky. Notice this lambda?

modify_time_sort = lambda f: os.stat("uploads/{}".format(f)).st_atime

So here's an example of sorting based on the access time that you get from the Linux stat command. The "st_atime" stands for "stat access time". That refers to the time the file was last accessed.

One other thing. We know that all those access times are in order of upload time because opening and closing files in /sendfile ensures that.

Let's Dump The Ugly Lambda

But my God this code is awful! That code would have been a lot clearer if I avoided the lambda. Something like this is nicer now that I think of it.

	def modify_time_sort(file_name):
	file_path = "uploads/{}".format(file_name)
	file_stats = os.stat(file_path)
	last_access_time = file_stats.st_atime
	return last_access_time

view raw better_modify_timesort.py hosted with ❤ by GitHub

Yeah, that looks better. Avoid cancer in your Python code. Say no to lambda boys and girls.

And Now We Have An End Result

With all being said and done, you now have a setup for uploading files that lists the files you uploaded. It should look a little like this.

FINAL CAVEAT: If you want to go public with something like this, read stuff on how to manage uploaded files in a secure way. Or let this be a starting point for a silly project that you have no intention of putting into prod. Oh, and here is a link to the Github project this article is based on.  Have fun!

When Side Projects Become Brownfields

Brownfield projects are where you maintain old code that's been around awhile.  Greenfields are projects where most of what you work on is fresh and new and innocent.  That is what "brownfield" and "greenfield" means to me.

Spearking of brownfields, suppose there's a pile of work to be done on some legacy code.  The source is ugly.  The technologies it relies on aren't that great.  You need a temporary escape.

Wouldn't it be nice to do a side project where you can pick your own tools and build something cool?  Then the realization kicks in.  This IS the side project.  This is what such projects become given enough time.

It Always Starts Out Innocently Enough

Flashmark was and still is my baby.  And yet, things aren't as ideal as they once seemed when first starting.  It was a greenfield.  Bootstrap was beautiful.  AngularJS was Spring dependency injection done right.  It was to be a single page application with JSON data and a nice API done up in Flask.

Then Reality Sets In

Experience and time served up some reality checks.  Angular turned out to be overkill.  There was nothing there that couldn't have been done easier with JQuery and Handlebars.  But the front end investment is in too deep at this point.

Sometimes, it is okay to accept bad infrastructure decisions that a project uses.  So I do.  Buyer's remorse, by itself, is an insufficient excuse for ripping out and replacing huge chunks of a code base.

For example, Flashmark had a new tagging system added recently.  Unfortunately, it required writing more Angular code.  This was done to make the new code look and feel like the code that's already there.  Consistency is a good thing.

What Makes It All Worth It?

It turns out some choices stood the test of time.  Bootstrap is still good for interface building.  JSON is still a good data format to work with.  Flask and SQL Alchemy are still awesome tools for building a back end API.  Worth it!

As success goes, a worthwhile side project doesn't necessarily have to make money or even have a user base outside of oneself.  Flashmark does neither.  All it does is make learning easier for me.  And that's good.  You can sum up my attitude with the following function...


def is_project_successful(user_count):
    return user_count >= 1


Working on a side project is a little like raising and caring for a pet.  It isn't always joy and bliss.  Things can get downright ugly at times.  But for the right project, the warts are worth it.  And if you stick with it long enough, you can still come out with something cool.  Happiness is a brownfield.