A Nice Usage of Laziness and Named Tuples in Python

When scraping sites, I sometimes find myself with upwards to a couple hundred html files in one local directory. By taking advantage of the lazy evaluation allowed via the yield syntax, it's really easy to process them all.

	def soup_line(dir_name, *exclusions):
	"""
	Pair up soups with the files they're based on.
	:param dir_name: Directory with the html files needed.
	:param exclusions: Don't include these files.
	:return: A tuple of soup file name pairs.
	"""
	import os
	from collections import namedtuple
	SoupFilePair = namedtuple("SoupFileName", ["soup", "file_name"])
	def is_valid_file(fpath):
	if os.path.isfile(fpath):
	if fpath.endswith(".html"):
	if fpath not in exclusions:
	return True
	else:
	return False
	for fname in os.listdir(dir_name):
	file_path = "{0}/{1}".format(dir_name, fname)
	if is_valid_file(file_path):
	with open(file_path, "rt", encoding="utf-8") as file_ob:
	soup = BeautifulSoup(file_ob.read().strip())
	yield SoupFilePair(soup, fname)

view raw soup_line.py hosted with ❤ by GitHub

A couple of things to note here. Since there is a yield statement present in soup_line, it behaves as a generator. One Beautiful Soup object is created and returned. Then it stops until the next loop around. This saves on a lot of time and loading of stuff into memory.

 I wanted a nice way to keep track of the names of the files associated with soup objects which called for some sort of tuple. SoupFilePair was the solution. This is an example of what's called a namedtuple. These objects behave just like regular tuples but can also double as a sort of "light-weight class". I like them because you don't have to remember as much what order the tuples were in that you were returning. Running "type" on the object will tell you what's what. It's sort of a nice bit of extra documentation whenever you need it.

	for soup, file_name in soup_line("sources"):
	print(file_name, soup("title"))

view raw use_of_soup_line.py hosted with ❤ by GitHub

I really like named tuples and laziness. It makes handling some of the bigger more complicated data beasties so much simpler to tame.

4 Ways to Awesomeness at Khan Academy Math

Khan Academy makes it possible to get good at a lot of math fast.  Being an open-ended site when it comes to learning it helps.  Here are 4 tricks that can help you get super far with it.

1.  Do Math When You Don't Have a Math Class.
Nobody doubts Khan Academy has helped a lot of people with their math grades.  However, doing it when you don't have any classroom obligations is awesome.  With no daytime curriculum to distract you, it frees you up to jump around at will.

I mix things up a lot.  I have no problem jumping around between algebra, geometry, calculus, and so on.  If geometry gets frustrating, I switch to something like statistics and come back to geom later.  I find it really helps refresh me and sometimes brings new perspectives.


2.  Set Your Own Goals.
There are a lot of good ways to set a goal but the point is that you should have something to shoot for.  Khan Academy gives you lots of reward badge opportunities.  While those are good, don't hesitate to get creative and make up your own goals

Here is how I do goal setting.  I have one wall in my study area where I have a bunch of Post-It notes with goals written on them and another wall to place the badges I won.  I give them cutesy little titles like you'd see on some X-Box achievement.  When I earn one, I stick a gold star on it and move it to my "win wall".  Looking at those badges and seeing what I've reached feels really good.


3.  Be Resourceful.
Sal Khan by himself does a really good job most of the time in his videos.  Just remember there are other good resources out there.  The discussion areas are helpful.  Looking at old problems you messed up on is a huge help.  We all learn in different ways so all I can say is do what works for you.  Trust yourself to know how to find the help you need.

For some cases, I find that just watching videos over and over and bombing on questions doesn't necessarily help by itself.  Sometime, I need detailed paper notes.  There are plenty of other Youtube channels that are helpful too.  Patrick JMT and Yay Math are examples of some I turn to that are stylistically different than Sal.  That's completely fine.  A fresh and different perspective is sometimes what's needed to get things to click.


4.  Take Care of Yourself.
Take a break.  Exercise.  Eat right.  Sleep.  Do things other than math with your life.  Walk away from the computer if you find math is getting too frustrating.    Go hang out with your friends.  There's nothing about math that's worth getting stressed and frustrated about.  Stress and poor health are bad.  Being relaxed, healthy, and alert gets you a lot further with your math endeavors.


Now Get To It!
There's only one warning.  Following these pointers might not just make you better at math.  You might actually find yourself enjoying it!  If you do become one of those people, that's okay.  I won't tell.