4-ch Thread Watcher in CL (4)

1 Name: #!/usr/bin/anonymous : 2009-08-28 02:48 ID:AXpao/1R

I always wanted a thread watcher for 4-ch. Something that keeps track of the threads I've replied to, and to inform me of new posts, and to allow me to remove/add threads to the watch-list of course.

So here it is, interactively from the lisp REPL:

(defparameter *watch-list* nil)

(defun watch-thread (url)
"Add the thread denoted by the url
or id to the watch list."
(dump
(let ((info (get-thread-info url)))
(unless (assoc (car info) *watch-list*)
(push info *watch-list*)))))


(defun extract-id (url)
(parse-integer (aref (nth-value 1 (cl-ppcre:scan-to-strings "pl/(.*)"
url)) 0)
:junk-allowed t))

(defun get-thread-info (url)
"Return a list with
CAR, thread ID
CADR, thread title
CADDR, board name
CADDDR, thread postcount
CADDDDR, thread link"
(let* ((text (aref (nth-value 1
(cl-ppcre:scan-to-strings "<h2>(.*)</h2>(.*)"
(drakma:http-request url)))
0))
(title (aref (nth-value 1
(cl-ppcre:scan-to-strings "(.*) <small>"
text))
0))
(postcount (aref (nth-value 1
(cl-ppcre:scan-to-strings "<small>.(.*).</small>"
text))
0))
(id (extract-id url))
(board (aref (nth-value 1
(cl-ppcre:scan-to-strings "4-ch.net/(.*)/kareha"
url))
0)))
(list id title board (parse-integer postcount)
(format nil "http://4-ch.net/~A/kareha.pl/~D"
board id))))

(defun unwatch-thread (id)
"Remove a thread from the watch list."
(setq *watch-list*
(delete id *watch-list* :key #'car)))


(defun update-list (&optional (list *watch-list*))
"Update all threads and print information of the new threads."
(dolist (x list)
(let ((new-info (get-thread-info
(format nil "http://4-ch.net/~A/kareha.pl/~D"
(caddr x)
(car x)))))
(if (> (cadddr new-info)
(cadddr x))
(progn
(setf (car x) (car new-info)
(cadr x) (cadr new-info)
(caddr x) (caddr new-info)
(cadddr x) (cadddr new-info)
(car (cddddr x)) (car (cddddr new-info)))
(dump (list x)))))
list))

(defun dump (&optional (list *watch-list*))
(format t "~{~{ID: ~D~%Title: ~A~%Board: ~A~%Postcount: ~D~%Link: ~A~%~%~}~}"
list)

Drakma and CL-PPCRE are required.
http://weitz.de/drakma/
http://weitz.de/cl-ppcre/

Find example uses here:
http://paste.lisp.org/display/86167
I haven't implemented a way to save/load the thread database yet, but that's very easy due to lisps READ/WRITE etc.

2 Name: #!/usr/bin/anonymous : 2009-08-28 02:51 ID:Heaven

Oh and in case it's not immediately obvious by the examples, it can keep track of many threads not a single one (all the examples use a single thread).

3 Name: #!/usr/bin/anonymous : 2009-08-28 14:05 ID:AXpao/1R

I implemented the save/load feature too. Add these lines to the source code:

(defparameter *list-path* "/foo/bar/4-ch.watchlist"))
(defun save-watch-list (&optional (path *list-path*)
(list *watch-list*))
(with-open-file (stream path :if-exists :supersede :direction :output)
(write list :stream stream)))
(defun load-watch-list (&optional (path *list-path*))
(with-open-file (stream path)
(dump (setq *watch-list*
(read stream)))))

Or if you can't be bothered, here's the whole source code, again:
http://paste.lisp.org/display/86187

Set *list-path* to the file you want the contents of your watch list to be saved, and use (save-watch-list) to save the watch-list, and (load-watch-list), to load it.

4 Name: #!/usr/bin/anonymous : 2009-08-28 21:56 ID:Heaven

The link in >>3 became invalid, and I've implemented a new version of the code. It's available here:
http://paste.lisp.org/+1UIH

Any further versions of the code will be added as annotations to this link.

This thread has been closed. You cannot post in this thread any longer.