url::encode

Comments on programming, web development,
and computers by Titus Stone.

Using Google’s jQuery CDN with Django when not in development

I wanted to transition the Django site I work with primarily to use Google’s jQuery CDN. However, when developing locally, it’s often faster to just use a local copy. What I wanted was a way to toggle which copy of jQuery was being used based on the environment.

Environment Detection
Before we can toggle the jQuery location, we need to have a way to detect which environment we’re running in.I previously blogged about how I have my settings.py configured. There are different ways to do this. One is to use local_settings.py, the other is to have conditional code in your main settings.py which determines values. Either way works.

As for my setting, on the webserver is a folder structure that resembles…

    \webroot
\django_devel
\django_prod
My webserver pulls double duty, hosting both a production version (“prod”) and a staging/testing version (“devel”). In my settings.py file I’m using this to determine which environment Django is running in…
import os
DJANGO_ROOT = os.path.abspath(os.path.dirname(__file__))

#
# Globals for determining settings
#
STAGING = PRODUCTION = DEVELOPMENT = False

if 'django_devel' in DJANGO_ROOT:
STAGING = True
elif 'django_prod' in DJANGO_ROOT:
PRODUCTION = True
else:
DEVELOPMENT = True

Most likely, you’ll have some other what you’re determining your environment. In that case just substitute that.

{% jquery template tag %}
Now that we have the required support code to detect what environment we’re running in, putting together the actual implementation is simple.

I decided to go with a template tag. It’s simple and easy.
import settings
from django import template
from django.conf import settings

register = template.Library()


# -----------------------------------------------------------------------------
# jQuery
# -----------------------------------------------------------------------------
@register.tag
def jquery(parser, token):
return JQueryNode()

class JQueryNode(template.Node):
def render(self, context):
jquery = 'http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js'
jquery_ui = 'http://ajax.googleapis.com/ajax/libs/jqueryui/1.7.2/jquery-ui.min.js'
if getattr(settings, 'DEVELOPMENT', True):
media_url = getattr(settings, 'MEDIA_URL', '/media/')
jquery = '%sjs/jquery-1.3.2.min.js' % media_url
jquery_ui = '%sjs/jquery-ui-1.7.2.custom.min.js' % media_url
return '<script type="text/javascript" src="%s"></script><script type="text/javascript" src="%s"></script>' % (jquery, jquery_ui)

There are a couple of things to note here.

First, there’s a hard-coded path in this code. Because I plan to always use this template tag every time I need jQuery, I felt confident doing so as it leaves only one place to edit. However, if you won’t be following that rigid of an implementation pulling the locations of jQuery out and into a settings.py file would probably be a smart idea.

Second, I’m implementing my type of detection for the environment. This probably differs from the majority of Django users.

When all is said and done, all that needs to remain is to simply drop the tag into base.html and we’re good to go.
{% load jquery %}{% jquery %}

How can I create a page that regularly updates itself with Django?

It seems this question has been coming up a lot on Stack Overflow, and I wrote a pretty lengthy response which I’ll include here. The original question is #1883266.

Q: I’m building a web application that needs to have content on the page updated in real time without a page refresh (like a chat or messaging application). How do I do this with Django? In other words, how do I use AJAX with Django?

A: There’s a lot going on in order to make this process work…
  • The client regularly polls the server for new chat entries
  • The server checks for and only replies with the newest
  • The client receives the newest entries and appends them to the DOM

This can be confusing when you’re first starting because it’s not always clear what the client does and what the server does, but if the large problem is broken down I think you’ll find it’s a simple process.

If the client is going to regularly poll the server for new chat entries, then the server (django) needs to have some type of API to do so. Your biggest decision will be what data type the server returns. You can choose from: rendered HTML, XML, YAML, or JSON. The lightest weight is JSON, and it’s supported by most of the major javascript frameworks (and django includes a JSON serializer since it’s that awesome).

# (models.py)
# Your model I'm assuming is something to the effect of...
class ChatLine(models.Model):
screenname = model.CharField(max_length=40)
value = models.CharField(max_length=100)
created = models.DateTimeField(default=datetime.now())

# (urls.py)
# A url pattern to match our API...
url(r'^api/latest-chat/(?P<seconds_old>\d+)/$',get_latest_chat),

# (views.py)
# A view to answer that URL
def get_latest_chat(request, seconds_old):
# Query comments since the past X seconds
chat_since = datetime.datetime.now() - datetime.timedelta(seconds=seconds_old)
chat = Chat.objects.filter(created__gte=comments_since)

# Return serialized data or whatever you're doing with it
return HttpResponse(simplejson.dumps(chat),mimetype='application/json')

So whenever we poll our API, we should get back something like this (JSON format)…
[
{
'value':'Hello World',
'created':'2009-12-10 14:56:11',
'screenname':'tstone'
},
{
'value':'And more cool Django-ness',
'created':'2009-12-10 14:58:49',
'screenname':'leethax0r1337'
},
]

On our actual page, we have a <div> tag which we’ll call <div id="chatbox"> which will hold whatever the incoming chat messages are. Our javascript simple needs to poll the server API that we created, check if there is a response, and then if there are items, append them to the chat box.

<!-- I'm assuming you're using jQuery -->
<script type="text/javascript">

LATEST_CHAT_URL = '{% url get_latest_chat 5 %}';

// On page start...
$(function() {
// Start a timer that will call our API at regular intervals
// The 2nd value is the time in milliseconds, so 5000 = 5 seconds
setTimeout(updateChat, 5000)
});

function updateChat() {
$.getJSON(LATEST_CHAT_URL, function(data){
// Enumerate JSON objects
$.each(data.items, function(i,item){
var newChatLine = $('<span class="chat"></span>');
newChatLine.append('<span class="user">' + item.screenname + '</span>');
newChatLine.append('<span class="text">' + item.text + '</span>');
$('#chatbox').append(newChatLine);
});
});
}

</script>

<div id="chatbox">
</div>

Efficiently View Apache Log Files on Windows

One of the annoying things about using multiple platforms is when one platform has a useful utility (no matter how small) and the other platform doesn’t. Have you ever needed to regularly check an Apache log file on your Windows development machine? The shell user inside me says “just tail it”… but this is Windows.

However, I just found a really amazing tool called BareTail.



BareTail “connects” to a log file, and shows you an automatically updated (live) tail of that file. It basically allows this situation to happen:

I have a window up on my developer machine (in the 2nd monitor) which is the tail of the Apache error log. Every time something is written to that log, BareTail pushes it into the window on my screen. I see log entries in real time.

Let me tell you, it makes debugging Apache error log issues much more efficient.

Time for Python Neenjah!

I happened upon an administrative assistant today who was renaming files using Windows Explorer. She had a folder with about 50 or so sub-folders, many of which contained sub-folders of their own. Every file needed to be renamed with the company name prefixing it. So for example, the file “january-charts.pdf” needed to be renamed to “Company - january-charts.pdf”.

“You know”, I said, “I could help that go a little faster if you’re interested.” I happened to know that these files needed to be sent as of yesterday, so I figured she wouldn’t mind me helping to trim an hour or two off them getting out. That must mean it’s time for Python Neenjah! (In case you missed it, that was a somewhat veiled reference to xkcd ‘Regular Expressions’)

A month or two ago I had put together a re-usable python module to allow easy recursively searching a directory. I know, I know, python already includes similar functionality. But it was weird to me, and I wanted a simpler and more flexible format. The module I built, inspire by some ideas I found around the internet, allows a callback to be specified whenever a file is found. It means you can do virtually anything from that directory search, and never really be concerned about how it does it.

from dirsearch import DirSearch

def search_callback(file):
print file

dir = DirSearch('C:\Path\Whatever', show_output=True)
dir.search(search_callback)

The source for dirsearch.py is included below.

Well with my module it only took a few lines of code to put together a command to complete the task at hand.

DirSearch.py


rrename.py

MarkEdit in Django

Being primarily a Django developer, it was only a matter of time until MarkEdit came with Django integration.

I added to the GitHub repo today a simple app which provides a widget that renders MarkEdit through Django. It’s probably a bit more difficult to use than the jQuery plugin because it makes some assumptions about your Django configuration and usage (as do many Django apps). In any case, an initiated Django developer should be able to figure it out just fine.

In addition to the source update, the wiki has also been updated with documentation for the Django integration.

An of course, here’s a screenshot of MarkEdit in the admin (running under Grappelli):

IE TextRanges, Selections, and Carriage Returns

I spent several hours this week trying to track down a bug with programmatically selecting text via javascript.


if (textarea.setSelectionRange) {
// Set selection for Mozilla-ish browsers
textarea.setSelectionRange(start, end);
}
else {
// Set selection for IE
var range = textarea.createTextRange();
range.collapse(true);
range.moveEnd('character', start);
range.moveStart('character', end);
range.select();
}
Assuming that ‘textarea’ is the element of the textarea, and ‘start’ and ‘end’ are Number values that indicate the start and end of the selection, respectively.

I was managing this in MarkEdit by keeping a state object that recorded the “Before Select” (everything before the selection), the selection text, and the “After Select” (everything after the selection).

So it seems easy to implement right?
start = beforeSelect.length;
end = start + select.length;
That doesn’t always work!

It works sometimes, but occasionally the selection will be the proper length, but randomly offset by 3-7 characters. It took quite a bit to figure out (that’s an understatement). Here’s what a lot of debugging and experimentation came up with:

When Internet Explorer creates a Range or a TextRange object, it converts all line returns to Carriage Return/Line Feed format; basically [0A] -> [0D 0A]. Easy to counter? Just determine all the instances of [0D] and [0A], subtract the two, then you have the offset? That’s what I thought too, but even with the offset, the discrepancy persisted.

So it was time for a new approach. What if we converted all of the [0A] characters in the textarea to [0D 0A]’s ? Again, the offset of the selection persisted.

Here’s what I finally narrowed it down to be: When you create a TextRange, IE doesn’t actually re-render the text internally, converting all [0A]’s. However, when you request the text (TextRange.text) it converts it at that time. This means if you were to get the length of the selected text, TextRange.text.length, it would be longer than what the TextRange is seeing internally! When using .moveEnd on the TextRange, the value counts the position from the internal text, not from the text it gives you.

That is reeeeeeediculously wrong. The final solution is to “sanitize” the text TextRange gives you by removing all [0D] characters from it. From there you can accurately calculate the selection position and give the TextRange the proper coordinates.

Whew… that was several hours needlessly wasted by IE (again).



MarkEdit Documentation

For those interested, I finished the preliminary documentation and sample code for MarkEdit. It can be viewed via the Bitbucket Wiki.

The documentation includes…

  • Installation
  • Sample code
  • Complete API reference
  • Complete configuration option reference

Developer Blog Entries

I haven’t done any development in the past few days thanks to this holiday weekend, however I did have a chance to catch up on my blog subscriptions (which were really beginning to pile up). My wife just peeked over my should and said I was also spending quality time with her.

  • Ryan Tomayko on Unicorn (ruby), “because it’s Unix.” A neat post on using some of the unixy functionalities of ruby to develop a network service. I’d really like to try it, but I don’t have a POSIX computer up and running. Hmmm… I should fix that.
  • Simon Willison talks about Node.js, a server-side javascript framework which looks ridiculously cool. Having been spending a lot of time working on MarkEdit, I can certainly appreciate some of the non-blocking/callback ideas that it implements. However, again with the lack of POSIX. It doesn’t yet run on Windows. I really need to remedy that.
  • Again from Simon Willison he mentions flXHR — A replacement for XmlHttpRequest, but implemented by an invisible flash (.swf) wrapper layer. It’s a really interesting concept, though completely hacky. Why bother with this? 1.) get around cross-browser issues, and 2.) get around browser security issues (like making an AJAX request to API on another domain for example).

WMD Editor & jQuery

I’ve been using Dana Robinson’s WMD Editor fork for a couple of months now. It was a great place to start. The problem is, it’s becoming a real pain to customize. Small things were taking huge amounts of time to accomplish so I had a tough decision to make: Continue to use WMD Editor and have to spend hours for customization that should only take a minute or two -OR- re-write the UI myself on top of the existing Markdown rendering component?

Introducing: MarkEdit — the Javascript MarkDown editor built on jQuery (to replace WMD Editor). It’s currently being hosted on BitBucket.

  • Built On jQuery/UI — Lighter weight code and less cross-browser issues
  • Themeroller Support — Use existing or custom jQuery UI Themeroller themes
  • International — Supports the loading of an alternate language
  • Sensible Defaults — You should be able to get up and running with 1 line of javascript
  • Configurable — Almost all defaults can be overridden very easily upon initialization
  • Public API — Interact with whatever part you’re interested in
  • Documented

Since everybody loves screenshots, here’s the editor running under a jQuery theme and using the Chinese language translation.





And again here’s a translated popup asking for the URL to insert an image…




And here’s preview mode using a different theme but still in Chinese…




So far I’ve invested about 15 hours into the project with it being probably around 95% complete for initial coding. There is still a TON of testing to be done. I haven’t even tried to fire it up in any of the fussy browsers yet.

I actually wrote a whole lot about it so far — it’s all up on the Wiki. There is still lots more documentation to come.