CoursePicker – The Saga Continues …

I’m working on a couple of enhancements to my CoursePicker web application to address some shortcomings of the original application.  On my current backlog are two major items:

1. Architecting the backend to handle multiple semesters. This enhancement is almost complete but I need to work on automating the process. Because I’m grabbing large CSV files and dumping them into my database, I ran into memory problems on dreamhost and had to use my local computer. A tentative solution is: splitting the larger CSV files and upload separately. This is a high priority item because during the available sections will change pretty frequently.
2. Redesign the webpage using my pitiful CSS skills. I’ve taken this to mean liberal use of divs and rounded corners. 🙂 This is coming along quite nicely and I’ll post some new screenshots in this post.
3. Restructure/refactor/consolidate code.

The Backend

The current production version of CoursePicker refreshes the database on a schedule (using cron). A bash script uses curl to grab the Fall 2013 csv file for the Athens campus from the Registrar Reporting site.

Then, this csv file is sorted; first according to the 3rd column which is the 4-letter course abbreviation and then on the section call number.

After the sorting process, the csv file is further processed using the cut [e.g. cut -d’,’ -f3-5 old.csv > new.csv ] command to create a new csv file with the wanted columns (columns 3 – 5). The output of this cut step is then fed into a PHP script which converts the data into a JSON file. Thereafter, this JSON file serves as the source of the course autocomplete field in the application.

Separately, the sorted csv file is then loaded into the database using a Java application which uses the mysql load statement to perform the database population step [e.g. -load data local infile ‘sorted_courses.csv’ into table Table fields terminated by ‘,’ enclosed by ‘\”‘ -]. Note: on Dreamhost, files larger than 6mb must be imported (as opposed to using phpMyAdmin to import the csv file).

Enhancements:

1. Instead of just getting 1 semester’s worth of courses, the new version will be pulling CSV files from the Athens and Gwinnett campuses. Note, the CSV files of interest begin with course_offering.
2. Since data from different semesters is being loaded into the same table, I need to ensure that any assumptions (e.g. that no call number is duplicated across Athens and Gwinnett campuses) still hold to prevent any bugs.

The Frontend

I’m limiting my application to the Athens and Gwinnett campus and the CSV files of interest begin with course_offering (it has over 20 fields). So, this frontend needed to be changed in response to this new requirement. However, I’ll describe the current production version’s setup (frontend) first and then describe the enhancements.

When the page is first loaded, a UserSchedule object is created and stored in the $_SESSION object. This UserSchedule is a fancy array which contains a unique id and a list of Section objects.

In the current/live version of CoursePicker, the user can narrow down their course selection through the official UGA requirements OR using a plain ole’ textbox which has autocompletion available. The end goal is: when the user selects a course (XXXX-2345), this value is sent to a controller (a PHP script) to retrieve the list of sections for that particular course. This list of Sections is returned to the View (web page) as a String which is converted to a valid JSON object and available for manipulating via JavaScript.

From here on, it’s trivial to add a Section object to the UserSchedule and there is a lot of DOM addition/insertion/deletion going on via JavaScript (coursepicker.js).

Enhancements:

1. Allow user to select the campus and semester of interest.
2. Remove the feature allowing users to drill down and simply allow them to search for courses by entering part of the course name or course number. Typehead.js allows you to specify tokens to be searched again so I’m using the course prefix (e.g. CSCI), the course number (e.g. 1302), and splitting up the abbreviated course name so that “Software Development” becomes “Software” “Development” and each word is now available as a token.”
3. Breakup coursepicker.js into themes e.g. a script to handle drawing items on the canvas, script to handle DOM modifications, etc
4. Breakup CSS similarly e.g. CSS that handles styling of my Section divs is one CSS file, styling the Canvas element is handled by another, etc.
5. Breakup controllers so that one controller handles getting items from the database and another deals with managing the UserSchedule object.
6. Make the user interface more “friendly”. This is definitely still a WIP (Work In Progress) but I think some changes I have in store will make things more userful. For instance, the live version of CoursePicker displays the list of sections for a chosen Course in a dropdown list. While this is great for courses with a long list of sections, it didn’t help my user plan better i.e. they had to click through all the sections to see the times, etc. Granted, I could alter the dropdown box to also display the times, but that would just be lazy. 🙂 So, I’m displaying each section as a styled div and allowing the user to hide or unhide the list of sections.

For the first frontend enhancement, I used the prefetch option in Twitter’s typeahead.js which caches the retrieved item in localStorage. The downside to this which I discovered was when the user selected a new semester, reloading the page with the new item didn’t work. I finally had to simple clear localStorage each time the user changed the selected item to something different and that forced the json file to be refetched.

$('#courseEntry').typeahead({
 name: 'courses',
 limit: 7,
 prefetch: $('#jsonURL').val(), 
 template: [ 
 '<p class="tt-courseShortname">{{coursePrefix}}-{{courseNumber}}</p>', 
 '<p class="tt-courseName">{{courseName}}</p>' 
 ].join(''), 
 engine: Hogan 
 }).
$('#jsonURL').val() //is a hidden input field which is updated whenever the selection changes

The first bonus for using typeahead.js is the ability to style the autocomplete entries! It’s pretty slick and once I get better at it, I’ll be sure to trick mine out even more. 🙂

The second is the fact that I can specify the actual value returned when the user chooses an item from the autocompete menu. In my case, a single autocomplete entry is composed of the course prefix, course number and the course short name but on selection, the course prefix concatenated to the course number is selected. The immediate benefit is that I can make my autocomplete suggestions much more descriptive (and more userfriendly) without making the application a hacky mess.

The third incentive from switching to typehead.js is the fact that instead of doing this to get the selected value, I can add an event listener to the object to get the value neatly.

The last reason (for me) for using typeahead.js is moving from over 50 LOC to less than 15 LOC to have autocomplete. Up until last week, I hadn’t been able to get typeahead.js to work. Part of this is due to the fact that I kept relying on my old (but working) code to get autocomplete working. However, last week, something changed: I read the README for typeahead.js. I mean I really sat down with it to understand what it was doing and once I did, it clicked. So I went from following the instructions here (https://gist.github.com/rn0/1848844) for remote JSON loading (if you use the typeahead module in Bootstrap 2.X) to something simpler and more elegant.

.on('typeahead:selected',function(obj,datum){
 var courseValue = datum.value; //Gets the XXXX-1234 returned by typeahead.js
 $.ajax({
     type: "POST",
     url: 'path/to/controller',
     data: { action : "getSections", courseEntry : courseValue},
     dataType: "json"
     })
 .done(function(msg){
     console.log("Success!");
 })
 .fail(function(msg){
     console.log(msg + "Error getting sections.");
 });

I didn’t mean for this post to get too long but I’m glad I’m finally semi-documenting how this project works. 🙂 Hopefully, my rewrite will be ready for the Spring semester craziness in a few weeks. As before, I’m putting the latest changes on Github but I’m aware that the readme sucks and doesn’t actually help you setup a replica of this app. However, this post should get you on the road to doing this yourself if you choose.

You can check out the work-in-progress here: the basic functionality (adding/deleting sections) has been added but the data is stale because I’m still testing.

Advertisement