Managing bookmarks becoming easier


I had mentioned last month that I am working on a bookmarker. Yeah I am still working and it is progressing. The search page is ready. I am using a work-around for adding the bookmarks and categories till the bookmark add page is ready. But I am happy with the work so far as it is becoming what I wanted although a bit slower due to lack of time. Here are some screenshots of working search page.

The search box has an autocomplete for giving suggestions for categories. It will give suggestions as per added categories. Pressing up down keys lets you navigate the categories and when you press enter the category is added on the page alongwith all bookmarks.

Below are some screenshots.

 

search-1 search-2 search-3

 

There is currently not much bookmarks added so only two categories came in autocomplete for search and one bookmark was added when category was chosen.

I forgot to mention earlier that the little cross in red lets you delete the category and the bookmarks tagged with the category. Obviously needed.

Advertisements

Migration from Python2 to 3 done. Working on a BookMarker


So I had decided that I will do the Python2 to 3 migration of my code base in April itself and I am glad that I was able to do it by 12th April. Solutions to codechef are still present but I decided to leave them be. If I restart online competitions again then I’ll think about them.

I have been thinking about making a BookMarker. I usually keep a lot of BookMarks and my bookmark bar becomes so messed up that I am just unable to arrange them properly. Yeah I like to keep things clean. I have folders inside folders inside in my bookmark bar… you can probably see how many bookmarks there are. It’s just that currently browsers do not allow good enough management of bookmarks. It gets tough. So I thought to make a bookmarker.

So how to manage bookmarks in a better way? Tags and proper search options. Let everything be flat(or in folders) and let there be tags for managing and searching Links. I understand that there are already many online ways to manage bookmarks but I like to try such things out. Also I have some other related ideas  so I thought I would be able to use them if I can control this. Why not try?

I have looked at different ways to make this – PySide, Kivy, django.

I have given up on PySide. I am totally fed up with the documentation that it has. Although there is a really good designer for making the UIs but still the lack of documentation totally puts me off. It is just a collection of links from over the internet so far.

Kivy is looking good and I was seriously considering to use it for this but as this is related to browser and using django will keep it closer to browser. Also a  simple startup script can start the browser with the bookmarker up and running.

Python 3 limits


I had decided that I will port my code from Python 2 to Python 3 and with the help of PyCharm the refactoring that I wanted to do was going completely smoothly. I decided in the middle to try something new. I thought when I was restarting Python then maybe try something new.

While doing this I hit Python 3 limits again. The library support is still not as good as for Python 2.  The first problem that I faced was with mechanize library. That was when I was trying to get some automation scripts done for logging into websites as per a schedule. Everyone wastes a lot of time online and I wanted to control it. Had the schedule but just opening websites by webbrowser module was not good enough. Why not use Python to login also? Mechanize was just what was needed. But it didn’t support Python 3. Shit. I found a solution to that. Using Selenium I did a test run and was able to login into http://projecteuler.net/ using my username and password. That I can work with now. But still mechanize might have been easier to work with.

Just 2 days back I had the idea of trying a bit of AI. Maybe a small bot or something? Natural Language processing for intelligent answers maybe? Maybe some simple  image recognition. Just the flight of thoughts that come and go every now and then. These are not like anything that I have tried before but what’s the problem with trying? I decided to go with Natural Language processing. Shit again. NLTK – the library needed for this had support for Python 2 only.

I was left thinking whether the advice that I got on Python mailing list for using Python 3 for new software actually good? Why actually did I change to Python 3? UNICODE is not something that I have even messed with once in Python. So why? I installed Python 2 but pip was missing. That’s a good thing that they did in Python 3,4, not that it actually helps with half of things not supporting Py3.4 yet.

Anyways I just noted down the ideas that came and decided to thing a bit more before going further. Hope someone reads it and decides to not go with “newest and latest” unless they have the time to port libraries as well.

Automating tasks in Python – Mass renaming of folders present in zip


So Python is getting useful to me for avoiding manual repetitive tasks. I used it to rename a lot of folders. There were around 700 folders in 28 zip files which I needed to rename and re-zip. That would have been really cumbersome if not for Python.

The folders’ name were like  0001 – abc 1 . Only that the numbers didn’t match. The first number was used for arrangement and it was the second number according to which I had to change the first number. Doing that for 700 folders would have been a real pain. Luckily I knew Python and wrote a script for doing this.
Here’s the Python script that I used.

import zipfile
import os
import shutil

CUR_DIR = os.getcwd()
FILES_IN_CUR_DIR = os.listdir(CUR_DIR)

def get_new_name(original_name):
    num = 0
    started= False
    for c in original_name[7:]:
        if not c.isdigit():
            if started is True:
                break
            continue
        else:
            started = True
            num *= 10
            num += int(c)
    return str(num).zfill(4) + original_name[4:]

def extract_files(zip_f, cur_path):
    with zipfile.ZipFile(zip_f, 'r') as zip_file:
        zip_file.extractall(cur_path)

def rename_files(cur_path):
    for files in os.listdir(cur_path):
        new_name = get_new_name(files)
        os.rename(os.path.join(cur_path, files),
                  os.path.join(cur_path, new_name))

def main():    
    files = [f for f in FILES_IN_CUR_DIR
             if f.endswith('.zip')]

    for f in files:
        cur_path = os.path.join(CUR_DIR, f[:-4])
        print('Extracting ' + f)
        extract_files(f, cur_path)
        print('Started renaming')
        rename_files(cur_path)
        print('Removing zip file ', f)
        os.remove(os.path.join(CUR_DIR, f))
        print('Writing back to ' + f)
        shutil.make_archive(cur_path, 'zip', cur_path)
        print('\n')if __name__ == "__main__":
    main()

This is just the first version that I used to get it to work. It creates temporary folders that I needed to delete but that was just one select and delete. Not a big thing. I plan to take care of the temporary folders too. The new one you can find in my github -> general -> Python33 -> Utilities maybe in a few days.

Making hierarchy of folders using Python


Suppose you want to make many folders aka directories for some purposes. Maybe you need to make many folders in a relative’s computers? Maybe you want to make many folders with same pattern in the names? Maybe someone has a habit of deleting files or formatting his hard drive and you have the responsibility of making a basic folder hierarchy>

I posted how to delete files so I thought to add one to make files also.

The hierarchy of directories that I am making is like this. Every indent means a subdirectory of the previous unindented one.

./project_euler
    ./001_050
    ./051_100
    ...
    ./401_450
./codechef
    ./easy
    ./medium
    ./hard
./spoj
./functions
./utilities

I made this script to learn Python mainly but it can be used to do same task of creating directories again and again easily. There are alternatives in Unix/Linux but in Windows it might be the easiest way.

#! python3
import os

TOP_LEVEL = ('spoj', 'functions', 'utilities', '_testing')
EULER_HIGHEST = 450
CODECHEF_FOL = ('easy', 'medium', 'hard')

def safe_make_folder(i):
    '''Makes a folder and its parent if not present'''
    try:
        os.makedirs(i)
    except:
        pass

def make_top_level():
    '''Makes folders with no subdirectories'''
    for i in TOP_LEVEL:
        safe_make_folder(i)

def make_euler_folders():
    '''Makes euler and its subdirectories'''
    for j in (os.path.join('project_euler', '{:03}_{:03}'.format(i, i + 49))
              for i in range(1,EULER_HIGHEST, 50)):
        safe_make_folder(j)

def make_codechef_folders():
    '''Makes codechef and its subdirectories'''
    for i in CODECHEF_FOL:
        safe_make_folder(os.path.join('codechef', i))

def main():
    make_top_level()
    make_euler_folders()
    make_codechef_folders()

if __name__ == "__main__":
    main()

This script’s comments are self-explanatory. Any questions? Just ask in the comments?

Opening websites using Python


Perhaps you need to open a set of websites for doing some work? Maybe opening facebook, twitter and other social networking websites? But you don’t want to click on many bookmarks or type the website’s name in your browser. Then this solution is for you. You can write different scripts for opening different set of websites. You can decide their sequence and even adjust the time between them.

Just change the names of websites in URLS, place them inside quotes ‘xyz_website’ and separate all the websites’ name by commas. Put any number in DELAY variable for the delay in seconds you want between opening of the webpages and you are good to go.

#! python3
import webbrowser
import time

URLS = [
    'http://codereview.stackexchange.com/',
    'http://programmers.stackexchange.com/',
    'http://stackoverflow.com/'
    ]
DELAY = 2

def main():
    for URL in URLS:
        webbrowser.open(URL, 2)
        time.sleep(DELAY)main()

Started writing a simple web crawler in Python for downloading a website


I have been having trouble looking through the C standard library so I thought to download the C library reference from cplusplus.com. I just wrote a simple script for downloading the front page and it is currently missing the style on the website but it got the basic contents. I’ll need to some work before it gives the best result.

I need to add code for recursively download everything but it’s a start.

Here’s my Python script.

#! python3
import urllib.request

filehandle = urllib.request.urlopen(
    'http://www.cplusplus.com/reference/clibrary/')

with open('test.html', 'w+b') as f:
    for line in filehandle:
        f.write(line)

filehandle.close()