JQuery Ideas That Help Me Memorize Really Big Numbers

The Idea

DomEditor is an app for editing a dominic number list.  Dominic numbers are a mnemonic device for memorizing really big numbers.  It works by tying every number from 00 through 99 with the image of people doing things.  Read more about the Dominic System here if interested.  My focus for this article is Javascript/JQuery techniques that helped with this project.

JQuery Event Delegation

Past projects had event handlers all over the place.  It was messy.  Here is an example from the RESTful notes project.

Behind every one of those "make button" functions is an event handler being created.  That's a lot of event handlers.  Not good.  Fortunately, JQuery has this thing called event delegation which you can read about here.  With this, you can use just one event handler at a parent level in the DOM.  There's no need to register separate handlers for every single button.  Here is how DomEditor does it.

The only catch to that is that DOM navigation is still awkward.  My solution was to do some pre-numbering on elements that needed it.  This setup gives me fuller use of Handlebars templates.  Worth it.

Promises, Closures, and Callbacks

Promises, closures, and callbacks matter because of all the async ajax happening here.  This creates a challenge.  The server-side API uses basic authentication for user-specific operations.  On the client, that means one call needed to get credentials and another call needed to use said credentials.  So how do you deal with that without the heavy nesting of callbacks inside of other callback?

Promises

Promises mean less needless callback nesting.  A call for credentials gets a promise for that info.  From there, you decide what to do with the data.  Here's how Dom Editor applies that idea....

Closure Callbacks

It gets weird trying to save info specific to one dominic number.  After all, the button callback only handles credentials.  The credentials have to be gotten from another call to begin with.  So how do you fit a dominic number argument into the picture?

The solution is to generate a callback.  Have it so the dominic number data to already there begin with.  Suddenly, argument passing is a non-issue.  That is straight up closure action.  Here's how that looks.

If this is all confusing, take heart.  Closures are hard to explain no matter what.  Really getting it requires tweaking existing ones and writing some of your own.  It should click after awhile.

Final Words

Get hands dirty with these ideas.  Write dumb junk pages that use delegate events if you have to.  Make promises.  Jump into that crazy closure world where functions return other functions.  It really is something special once you get these concepts.  You'll be a better programmer for it.  You'll probably also find better uses for these ideas than memorizing a hundred digits of PI.

Making an HTML5 Paint Program For Fun And .... Fun

Here's the project idea.  SVG Paint is an HTML5 art application.  User logs in.  User doodles with different brush types of shape, size, and color.  User can save and load their drawings.  Pretty straight forward.

Now for some thoughts and principles about what went into this project.

Keep The User Interface Simple

The user shouldn't have to think.  Everything in a layout should be straight forward, natural, and familiar to the user.  Here's my take on that.

Logging in.  You start with a login page.  You create a user account with a password.  If everything goes well, you can log in with the new account.  If you messed up the password or tried making an account that already exists, it'll tell you.

Here's what that looks like.

When you log in to the actual paint program itself, it's time to get creative.  After having some fun with it, you might get something like this.

Loading and saving are done through the file menu.  It will load drawings and save them based on the name entered in the "File Name" field.

And that's it.  It's simple, straight forward, and doesn't try to be clever.

Design Good Systems For Handling User Data

Data is everywhere.  It's on the front end.  It's in the database.  It's everywhere in between.  Funny enough, designing a system for getting data from point A to point B is just as much of an art as laying out Bootstrap elements and graphics.  Here's my take on dealing with data for this project.

The front end for making and logging in users is all Ajax and JQuery.  It communicates user id and passwords over plain HTTP using basic authentication.  The back end is done in Flask with an extension called Flask-Login. Passwords are stored and matched using a SHA1 hasher.  Verifying those encrypted passwords ends up looking like this.



Basic authentication on the client side also must be dealt with by the API on the server side.  That helps in managing user-specific data.  This setup was somewhat inspired by Github's API.  It's simplicity is reflected in the fact that the Requests library uses that API as an example in it's documentation.  Flask makes it easy to follow Github's example.  Take saving a drawing for instance.



Loading and saving the drawing data was interesting in and of itself.  The SVG is XML embedded within the DOMhierarchy. That XML gets converted to and from JSON.  SVG.js was used to create the svg data that make up the image on the canvas. On the server side, the data relationships were such that they played very nicely with the SQLAlchemy take on ORM mappings.

Be Critical Of Your Own Work

A critical eye matters, especially for one's own creative works.  Here are some issues about this project that make it less than perfect.

Security Problems
Security features are lacking in this project.  There are no rules for passwords.  Password encryption doesn't use salts.  My ignorance on which hasher algorithm fits the bill probably matters.   Understanding of Flask-Login is hampered by docs that partly depend on already knowing WTForms.  Lastly, usage of HTTP for the api is a less secure communication channel than HTTPS would be.  This project is not something that belongs on a public facing production server.

Data Communication Ineffeciency
It also probably isn't the most efficient data setup in the world.  It only takes a few minutes of doodling to generate tens of thousands of data records.  That means huge data payloads every time a load or save happens.  This strikes me as impractical.

Making Things Is Its Own Reward

All that being said, I had fun with this.  It isn't useful but it's not really trying to be.  Bored programmer has some stupid fun with a project and shares it with the world.  Worth it.  Everybody should do this.

Stepping Backwards From AngularJS to jQuery - An Experiment

Do Javascript frameworks like AngularJS matter that much when it comes to single page applications?  After more than a year of ups and downs with Angular, it was time for me to explore this question.  To do so, I wrote a little note application.  This was going to be done with just jQuery with Handlebars.js being added in later.  Here are some thoughts about that experience.

The Ajax Aspect

Ajax is important to much of what I do in my projects.  Headers and JSON get sent back and forth a lot.  Also, there are plenty of callbacks for dealing with stuff from the server.

So how do Angular and jQuery compare in that department?  To be honest, you don't lose much.  You go without Angular's $http service but you still have jQuery's ajax function to work with.  Here's what a json-in-json-out scenario looks like in AngularJS.



Here's the same thing but done with jQuery.


Overall, the only significant feature here is that $http has to be dependency injected.  I don't know that I gain anything from grabbing a service from out of whatever container it happens to comes from.  It feels vaguely like working with a Spring container in an enterprise Java project.  But that's about it for me.

Know Thy DOM

Giving up Angular did force me to think more about the DOM.  Vanilla jQuery doesn't insert itself as readily into HTML code as an Angular directive can.  That fact made direct DOM dealings more important.  For examples, here's how you generate bullet point list in Angular.

Doing the same thing in jQuery, you get something that looks like this.

While trivial in this example, string concatenation can gets messy as the HTML gets more involved.  However, you do get that upshot in jQuery of having debugger features more conveniently available.  Setting a breakpoint in straight Javascript is easy.  I have no idea how to do that with an ng-repeat loop.

Filling The Template Void with Handlebars.js

The good news is that living without Angular doesn't mean giving up on client-side templates entirely.  It turns out that Handlebars.js fills that void nicely.  Drop a template between script tags.  Load it up.  Insert the data.  All is well.


Yes, there is DOM dirty work to do but that's okay.  At least there isn't that error-prone process of piecing strings together so much.  That's actually pretty nice.

My Dynamic Button Trip Up

One area that tripped me up with Handlebars was button handling.  Buttons for deleting and changing notes are specific to each note.  The event handlers for those buttons need to reflect that.

Handlebars is really good at taking data text and plugging it into template text.  Where it gets weird is when events come into the picture.

The thing is this.  You generally don't attach event handlers directly to the text that Handlebars generates.  You attach them to some kind of DOM object like this...

$("#myButton").click(function(e){})

Okay, so what to do?

Here was my solution.  Let Handlebars do the whole template thing.  Have jQuery wrap the text up as a DOM element.  Then, just let jQuery attach event handlers to that.  Here is what that code ended up looking like.

So that gets the job done.  Once the DOM is created, we can attach a DOM button to that structure.  And that's it.  In case you're curious, you can find the full js file where this is set up over here.  And if I'm "doing it wrong", feel free to yell at me in the comments section.

Final Thoughts

Going without Angular was nowhere near as hard as I thought it would be.  The Ajax parts are reasonable thanks to the core jQuery libraries.  The DOM aspects I found myself needing to handle really weren't that burdensome.  And thanks to Handlebars, I didn't even have to give up templates.

What Angular is good for is catering to my enterprise and Java background.  I've done my share of mixing logic and HTML together from my experiences in JSP and ColdFusion.  The dependency injection that happens in Angular feels like a simplified version of what happens in a lot of Spring projects.

All being said, I don't think I'm going to disavow AngularJS entirely because I'm still not convinced that it's a "bad" framework.  For bigger web applications, it might well be ideal for all I know.  It's just that I now know that the set of problems Angular fits for isn't as big as I thought.  And that's an okay thing to come to terms with.  Here's to the programmer's journey!

Python and Javascript Jam Session - A Web Interface for Apt-Cache

Just fun fun.... How easy is it be to make a web interface over the apt suite of tools?  That's what AptPackageShow seeks to explore.  This post will cover aspects of this project that I personally found to be interesting.

Technically, this is not a FUMBLINA project.  The Debian packaging system is already a database of sorts.  No need to drag MySQL into it.  So scratch the M out of the acronym for this project.  No biggie.  There's still plenty of neat stuff to check out.

Building The API

So the first thing to fire up a Python editor to put together an api.  Starting with a model.py file means a short path from coding to seeing actual results.  Said results might be in a terminal shell but that still counts for something.  

For this model interface, we'll work with parts of apt-cache.  The "search", "show", and "stats" sub-commands are of interest here so we build functions to interface with them.  

A lot of the implementation details of the api is just Flask boilerplate and manipulating of string, dictionaries, and lists.  It's fun to do but, to be honest, it's pretty boring to talk about.  The most I can say on that matter is don't do things like this....

That's pretty bad.  If you must write code like this, make sure you refactor it ASAP.  Future You will thank you for it.

Working The Subprocess Module

The check_call function from the subprocess module is a key ingredient for this project.  Here's an example of this....

There are a few takeaways from this experience with invoking outside processes in this way.

1. If you have to make calls in a context where you'd normally need shell access, set the shell=True option.
2. Using absolute paths for commands simplifies things.
3. If you deploy to a test server, make sure the running Python process uses an account with shell permissions.

UPDATE:

Someone on Reddit pointed out the shell=True thing in point 1 isn't true.  Setting shell=False actually does turn out to work just fine.  You only need make it True if you run commands that are shell specific.

One last thought on this subtopic.  Running something like this in production feels wrong.  We are allowing shell access to a process that gets information directly from the user.  That strikes me as kind of a bad thing.  So in no way should this be considered an example of secure code.

Building Up An AngularJS Interface

Time to switch from Python to Javascript.  There's this idea that a component style to Angular code means fewer headaches with respect to where your data is available.  This isn't wrong but it isn't completely true either.  For example, component hierarchy for this project looks like this.....

And then there is the thing about having to wire together variables like this....

From there, you're supposed to pass info on like this....

So there's a lot of wiring to think about when you trade your scope hierarchies for component hierarchies.  Those can still be painful in the same way that ANY code hierarchy is painful.  That's why "search results" and "package details" are part of the same component.  It helps keep the hierarchy flatter.

Last Thoughts

What's the point of all this?  This was a fun diversion mainly done out of curiosity in the "What does this button do?" sense.  Having gotten practical experience with Angular components from this, I regret nothing.  I hope you enjoyed reading this little spiel about my project.  Again, go here if you want to see the project for itself and/or play with it.  Have fun!

Making Things Within The Quadrants of Want

"The way to get startup ideas is not to try to think of startup ideas. It's to look for problems, preferably problems you have yourself.  The very best startup ideas tend to have three things in common: they're something the founders themselves want, that they themselves can build, and that few others realize are worth doing. Microsoft, Apple, Yahoo, Google, and Facebook all began this way." - Paul Graham - How to Get Startup Ideas

My motivations for Flashmark were based on this essay.  It is a tool aimed at my learning issues.  I wanted this for myself.  However, recent attempts to market it drew little interest.  Nobody tried it.  Nobody commented.  I'm mostly okay with this!  But it did get me thinking of the whole idea of building software people want.

The Four Quadrants Of Want

There are four kinds of "want" when it comes to a software product.

Quadrants Of Want

Quadrant 1.) The developer wants it and the users want it.

This is ideal.  The developer is a user and other people use it too.  Mark Zuckerberg has a Facebook account as do over a billion other people.  There is also the case of the smaller developer who uses her own SaaS product but also has a couple hundred paid subscribers.  Whatever the scope of your user base is, this is the best quadrant to be in.

Quadrant 2.) The developer doesn't want it but the users do want it.

This isn't a bad deal either.  You have no dog in the fight when it comes to the product.  Still, users like it enough to pay which means you get paid too.  I've worked on software for archiving airplane part drawings and tracking phone tolls in office telephone systems.  I have no personal interest in planes or phones.  Still, the process of working on these things was enjoyable and co-workers were awesome.  These were good gigs.

Quadrant 3.) The developer wants it but the users do not want it.

This has happened for me twice with Flashmark being my most recent instance of this.  In the past, the Uncle Squirrely was a project I did that notified me of videogames that just dropped to "greatest hit" price levels.  In neither case did these projects get anything in the way of a user base.  But I didn't care.  I was scratching my own itch.  It didn't matter whether or not others had an itch in that same spot.

Quadrant 4.) The developer does not want it and the users do not want it either.

This is what Paul Graham wants people to avoid.  It happens when you don't want the product but you THINK other people will buy into it.  When they don't, the end result feels like a waste of time.  Pure demoralization.  Graham felt it first hand and would rather other startups not be crushed by that same defeat.

Moving Between Quadrants

When you start in quadrant 3, you are guaranteed a user.  It is you.  You are a worthy customer base to develop a quality product for.  Know it.  Believe it.  Even if the product never leaves this zone, you are at least spared the quadrant 4 nightmare scenario.

Quadrant 3 projects can serve as potential gateways to quadrants 1 and 2 if you allow it.  The key thing here is to be open.  Share what you are do as early and often as possible.  This allows for feedback for improvements.  It could lead to a new job.  It could make your project profitable.  But most importantly, feedback makes projects more fun.  You'd be out of your mind to not want to up the fun factor!

Practicing What I Preach

Don't do what I did with Flashmark.  There was very little communication with other people during that project's development.  Only two release announcements were ever made about it.  Both of them were made when the project was close to "done".  The end result is still a success.  However, I will admit it would have been nice if it were a different kind of success.

Instead, do something like this..... Last week, I started a just-for-fun project to put up a web interface to the apt-cache tool commonly found on Debian and Ubuntu systems.  I'm doing it because I'm curious.  How easy would it be to build a small AngularJS project using a component-based approach?  How would it work out if I put a Flask API on top of a command-line tool?  That's what I'm interested in finding out with the AptPackageShow project.  It's incomplete but you can go here if you want to check it out.  Comments welcome!

See the difference there?  With AptPackageShow, the conversation opens up almost right away instead of the "not talking about it much" approach taken with Flashmark.  I'm going to go out on a limb in saying that this is a better way.

Flashmark - What It Is And Why I Made It

Consider this an experiment or perhaps a tech demo if you will. The idea here is to see how other people react to a project built with the FUMBLINA tech stack.  It's not perfect.  Eventually though, one has to expose a work to the world and see what kind of reaction it gets.  So here goes nothing.

What Flashmark Is

Flashmark makes learning easier by mixing together the ideas of a flashcard and a bookmark.  First, you create a flashcard.  Second, you attach bookmarks to that flashcard.  That's a flashmark.

Here is what a list of flashmarks looks like.

Below is a question and answer for one of them.

This is a what an attached learning resource looks like.

Flashmark Q and A should be short as should the bookmarks also be.  The 140 character limit imposed on everything enforces that.

It's been tested on Chrome on a Linux desktop and Chrome for Android on a Samsung Galaxy phone.  It should work well on anything that plays well with AngularJS and Bootstrap.  If it fails on your platform, leave an issue here.

Personal Testimonial

I built this because I like teaching myself stuff.  So here is my use case for this project.

Every morning, I train myself using Anki flashcards.  If I do bad on a card, a note goes on the whiteboard and I move on to the next one.  This continues until my daily review allotment is done.

Afterwards is my "attack the weakness" phase.  This is where Flashmark comes in.  New flashmark questions and answers get made for whatever I put on the board.  I google up relevant links and attach them to the new cards.  Ideally, each card will get more than one bookmark if I can help it.

Mobile-friendliness is a sweet bonus.  Today, for example, a grocery line wait gave me a moment to review worker connection setup for Nginx configuration.  It's a great time optimizer as studying goes.

My Anki deck statistics for software development got a boost from this project.  Right now, my current "mastery" is at 98%.  This is for a collection of about 700 cards.  Not bad!

Some caveats to my results are due however.  For one, a 98% self assessment is no substitute for outside parties reviewing one's work.  Also, if I had to be completely honest with myself, some skill gain was likely the result of building Flashmark itself.  So I naturally have reason to be skeptical of my own success with this software.

Feel Free To Try It

Go right on ahead and try Flashmark for yourself.  Comments and criticism are welcome.  The reactions, good or bad, should hopefully be instructional.

Using Python To Play With Binary Files

Suppose you were to open up an image file in Python 3. Even without knowing much about image formats or binary files, what's possible here?  Let's see what we can find out about these files.  Let's also see what it says about learning strategies.

Some General Ideas

Reading binary files isn't much different than reading text.  One difference is that you open it with "rb" for "read binary" instead of "rt" for "read text".  If the file in question is small enough, you can grab the data with a one-liner like this....

raw_data = open(file_path, "rb").read()

One issue with managing bytes is that there is sometimes text in the mix of it worth considering.  Normal text operations often won't play well with bytes.  For that reason, we decode the bytes so that we have actual text strings to work with.

text_data = byte_data.decode("ascii", errors="ignore")

And yes, a lot of data in the source binary won't translate nicely to text.  Right now, we don't care about that stuff so we ignore it.  

What about going the other direction?  Sometimes, a text string needs to be a pile of bytes.  So we want to encode this.

byte_data = text_data.encode("ascii")

Do you need to do encoding and decoding everywhere you go?  Not necessarily.  It's just a generally good thing to understand the idea of working in both directions. Now, with those ideas established, let's poke around with some image files.

Identifying Image File Formats (sort of)

Here's a thought.  You can dump bytes from an image file straight to the console.  A good Linux shell environment is handy like that. Heck, you could even page through the mess if you felt so inclined.  For example, you can do this.....

cat pinkie1.png | less

From that, you get something that looks like this....

While most of the data isn't readable, there are bits of text there.  Also to note is that PNG, GIF, and JFIF (aka JPEG) appear to have something in common.  The first instance of text that shows up seems to consistently identify the image format.  So that's useful.

Finding text looks like a job for regex.  But wait a minute!  This is bytes.  Regex is meant to be used with text.  No problem.  We'll just decode the bytes we can and toss the rest.  The result is a function that looks like this.

So if you put in image_format("pinkie1.png"), you get PNG as the result.  Pretty nifty.

But also pretty useless.  This "pick-the-first-ascii-text-block-you-find" approach has only been seem to work with samples of PNG, GIF, and JPEG files.  It bombs on TIFF, Windows BMP, and probably several other image formats too.  Oh well.

Stirring Up Curiosity Is Its Own Reward

Making a turd like this one did get me thinking.  How would the pros do it?  There is this function in the official Python 3 libraries called what from the imghdr module.  How, exactly, did they do it?  Oh wait!  There is source code out there.  Here's an unofficial Github resource right here.  This implementation is good reading and beautiful in its simplicity.

This is a good learning strategy for a lot of things.  You poke around with no particular goal.  You stumble upon something.  You dive in.  You screw it up.  You step back. You see how the experts handle it.  Lather.  Rinse.  Repeat.  It's both educational and a lot of fun.  So go do that!