squash merge feature: tagline

Makefile

@@ -1,15 +0,0 @@
-
-
-all: template.html content.txt timeline.css
-	python microblog.py ./template.html ./content.txt > result.html
-
-# for people who don't want to read the README 
-# and want to hit `make` to see how things work.
-template.html:
-	cp ./example/template-generic.html ./template.html
-
-timeline.css:
-	cp ./example/timeline-generic.css ./timeline.css
-
-content.txt:
-	cp ./example/demo.txt ./content.txt

README.md

@@ -5,19 +5,33 @@ Simple and stylish text-to-html microblog generator.
 
 ## Requirements
 
-    python3 make date
+    python3 date make curl pycurl urllib
 
-`date` is `date` from GNU Core Utilities.
+`date` is `date` from GNU Core Utilities. `make` is only used to demonstrate the examples in `example/`. The latter three are optional; `curl`, `pycurl` and `urllib` are only used for the uploader script.
 
-## Usage
+### Usage
 
-Invoke `make`.
+Send three arguments minimum to `python`. The fourth argument for an e-mail address is optional.
 
-This script reads content from a text file and applies the content to an html template. It will format hashtags, links, and images automatically. The template must contain three placeholders, in order, for the post count, tag cloud, and timeline content.
+    python microblog.py ./template.html ./content.txt user@mailhost.tld
+
+The resulting web page is outputted from standard output. Therefore:
+
+    python microblog.py ./template.html ./content.txt user@mailhost.tld >  result.html
+
+Use a Makefile (or another script) to simplify invocation.
+
+    cp example/Makefile .
+
+This script generates three text files after operation.
+
+* `postsperpage.txt`, holds an integer for the number of posts to render per page (default: 20). Thi is a configuration file that is created if it does not exist).
+* `lastfullpage.txt`, holds an integer for the last page rendered by the paginator.
+* `updatedfiles.txt`, a list of files updated by the script for use in automated uploads.
 
 ### Writing Content
 
-See example.txt.
+See `example/demo.txt`.
 
 The content file is a plain text file of posts. Each post has two types of information: timestamp and message. For example:
 
@@ -37,18 +51,6 @@ The content file is a plain text file of posts. Each post has two types of infor
 * the two last lines of the file must be empty
 * html can be placed in the message for embedded videos and rich text
 
-### Generation
-
-Send three arguments minimum to `python`. The fourth argument for an e-mail address is optional.
-
-    python microblog.py ./template.html ./content.txt user@mailhost.tld
-
-The resulting web page is outputted from standard output. Therefore:
-
-    python microblog.py ./template.html ./content.txt user@mailhost.tld >  result.html
-
-Use the Makefile (or another script) to simplify invocation.
-
 ## Anything else
 
 This is a script I wrote for personal use. The output can be seen on [https://likho.neocities.org/microblog/index.html](https://likho.neocities.org/microblog/index.html). I figure someone else may want to use it for their own personal websites, so it is published. 

example/Makefile

@@ -0,0 +1,22 @@
+
+
+all: template.tpl content.txt timeline.css
+	python microblog.py ./template.tpl ./content.txt > result.html
+
+# for people who don't want to read the README 
+# and want to hit `make` to see how things work.
+template.tpl:
+	cp ./example/default.tpl ./template.tpl
+
+timeline.css:
+	cp ./example/timeline.css ./timeline.css
+
+content.txt:
+	cp ./example/demo.txt ./content.txt
+
+.PHONY: clean
+clean: 
+	rm ./pages/*.html 
+	rm ./tags/*/*.html 
+	rm lastfullpage.txt
+	rmdir ./pages ./tags/* ./tags

example/default.tpl

@@ -13,25 +13,30 @@
 
 <h1>A Microblog in Plain HTML</h1>
 
-<div class = "column">
+<div class = "row"> <div class = "column">
     <div class="profile">
         <img src="./images/avatar.jpg" alt="Avatar" class="avatar">
         <span class="handle">Your Name Here</span>
         <p><span class="email"><a href="mailto:user@host.tld">user@host.tld</a></span></p>
-        <div class="bio">Description</h4>
+        <div class="bio">Description
-            <h4>%s total posts</h4>
+            <h4>{postcount} total posts</h4>
             <h3>Tags</h3>
-            <p>%s</p>
+            <p>{tags}</p>
             <h3>Pages</h3>
-            <p>%s</p>
+            <p>{pages}</p>
         </div>
     </div>
 </div>
 
-<div class = "timeline">
+    <div class = "timeline">
-%s
+    {timeline}
+    </div>
 </div>
 
+<center>
+<a href="https://notabug.org/likho/microblog.py">microblog.py</a>
+</center>
+
 </div>
 </body>
 </html>

example/timeline.css

@@ -52,11 +52,23 @@
     margin: 1em
 }
 .gallery {
-    margin-left: auto ;
+    margin:auto;
-    margin-right: auto ;
     display: flex;
     align-items: center;
-    max-width: 100%;
+    width: 100%;
+}
+div.panel {
+    margin: 2px;
+    width: auto
+}
+div.panel img {
+    width: 100%;
+    height: 100%;
+}
+
+div.panel img:hover {
+    border: 1px solid #777;
+    filter: invert(100%);
 }
 .header {
     background: black;
@@ -65,21 +77,9 @@
     font-weight: bold;
     padding: 0.5em;
 }
-div.panel {
+/* Clear floats after the columns */
-    margin: 2px;
+.row:after {
-    /* border: 1px solid #ffbc06; */
+    content: "";
-    width: auto
+    display: table;
-}
+    clear: both;
-
-div.panel:hover {
-    border: 1px solid #777;
-}
-
-div.panel img {
-    width: 100%;
-    height: 100%;
-}
-
-div.panel img:hover {
-    filter: invert(100%);
 }

microblog.py

@@ -1,24 +1,19 @@
 
-import sys
+import sys, os, subprocess
-import subprocess
-import os
 
+# apply div classes for use with .css
 def make_post(num, timestamp, email, msg):
-    # used <a class> for name and email but it's not actually needed.
-    #     <a class=\"name\">%s</a>
-    #     <a class=\"email\">(%s)</a>
     output = ""
-    fmt  = ""
+    fmt  = '''
-    part = '''
 <div class=\"postcell\" id=\"%i\">
     <div class=\"timestamp\"><a href=#%i>%s</a></div>
     <div class=\"message\">%s</div>
 '''
     if email != None: 
-        fmt = part + "<div class=\"reply\"><a href=\"mailto:%s?subject=p%i\">[reply]</a></div></div>"
+        fmt += "<div class=\"reply\"><a href=\"mailto:%s?subject=p%i\">[reply]</a></div></div>"
         output = fmt % (num, num, timestamp, msg, email, num)
     else:
-        fmt += part + "</div>"
+        fmt += "</div>"
         output = fmt % (num, num, timestamp, msg)
     return output
 
@@ -31,13 +26,15 @@ def make_gallery(i, w):
         image = w.pop(index)
         template = '''
 <div class=\"panel\">
-    <a href=\"%s\"><img src=\"%s\" class=\"embed\"></img></a>
+    <a href=\"%s\"><img src=\"%s\" class=\"embed\"></a>
 </div>
 ''' 
         tag.append(template % (image, image))
     tag.append("</div>")
     return tag
 
+# apply basic HTML formatting - only div class here is gallery
+from html import escape
 def markup(msg):
     result = 0
     tagged = ""
@@ -51,10 +48,12 @@ def markup(msg):
         if word.find("src=") == 0 or word.find("href=") == 0:
             continue
         elif word.find("https://") != -1: 
-            new_word = ("<a href=\"%s\">%s</a>") % (word, word)
+            w = escape(word)
+            new_word = ("<a href=\"%s\">%s</a>") % (w, w)
             words[i] = new_word
         elif word.find("#") != -1 and len(word) > 1:
             # split by unicode blank character if present
+            # allows tagging such as #fanfic|tion
             w = word.split(chr(8206)) 
             # w[0] is the portion closest to the #
             tags.append(w[0])
@@ -69,75 +68,106 @@ def markup(msg):
     words += gallery
     return " ".join(words), tags
 
-def parse_txt(filename):
+# iterate through posts and get information about them
-    content   = []
+def get_posts(filename, email):
-    with open(filename, 'r') as f:
+    class Post:
-        content = f.readlines()
+        def __init__(self, ts, msg):
-    stack   = [] # stack of lists - only a stack b/c order of contents
+            self.timestamp = ts # string
-    message = [] # queue of lines
+            self.message = msg  # list
-    #  {-1 = init;; 0 = timestamp is next, 1 = message is next}
+        pass
-    state       = -1 
-    timestamp   = ""
-    for line in content:
-       # print (line, len(line), state)
-        if state == -1:
-            state = 0
-            continue
-        elif state == 0:
-            cmd      = ['date', '-d', line, '+%y %b %d']
-            result    = subprocess.run(cmd, stdout=subprocess.PIPE)
-            timestamp = result.stdout.decode('utf-8')
-            state     = 1
-        elif state == 1:
-            if len(line) > 1:
-                message.append(line)
-            else:
-                stack.append([timestamp, "<br>".join(message)])
-                message = []
-                state = 0 
-    return stack
 
-def make_timeline(filename, email=None):
+    def parse_txt(filename):
-    timeline  = []
+        content   = []
-    tagcloud  = dict()
+        with open(filename, 'r') as f:
-    # note: content is ordered from newest to oldest
+            content = f.readlines()
-    # format requires two line breaks before EOF
+        posts   = [] # list of posts - same order as file
-    stack_of_posts = parse_txt(filename) # [timestamp, message]
+        message = [] # list of lines
-    count = len(stack_of_posts)
+        #  {-1 = init;; 0 = timestamp is next, 1 = message is next}
-    for post in stack_of_posts:
+        state       = -1 
-        p, t = markup(post[1])
+        timestamp   = ""
-        timeline.append(
+        for line in content:
-            make_post(count, post[0], email, p)
+            if state == -1:
-        )
+                state = 0
+                continue
+            elif state == 0:
+                cmd      = ['date', '-d', line, '+%y %b %d']
+                result    = subprocess.run(cmd, stdout=subprocess.PIPE)
+                timestamp = result.stdout.decode('utf-8')
+                state     = 1
+            elif state == 1:
+                if len(line) > 1:
+                    message.append(line)
+                else:
+                    p = Post(timestamp, "<br>".join(message))
+                    posts.append(p)
+                    # reset
+                    message = []
+                    state = 0 
+        return posts
+
+    posts = parse_txt(filename)
+    taginfos = []
+    tagcloud = dict() # (tag, count)
+    tagged   = dict() # (tag, index of message)
+    total = len(posts)
+    count = total
+    index = count # - 1
+    timeline = []
+    for post in posts:
+        markedup, tags = markup(post.message)
         count -= 1
-        for tag in t:
+        index -= 1
+        timeline.append(
+            make_post(count, post.timestamp, email, markedup)
+        )
+        for tag in tags:
             if tagcloud.get(tag) == None:
                 tagcloud[tag] = 0
             tagcloud[tag] += 1
-    return timeline, tagcloud
+            if tagged.get(tag) == None:
-    pass
+                tagged[tag] = []
+            tagged[tag].append(index)
+    return timeline, tagcloud, tagged
 
-def make_tagcloud(d):
+def make_tagcloud(d, rell):
     sorted_d = {k: v for k, 
                 v in sorted(d.items(), 
                 key=lambda item: -item[1])}
     output = []
-    fmt    = "<span class=\"hashtag\">%s(%i)</span>"
+    fmt = "<span class=\"hashtag\"><a href=\"%s\">%s(%i)</a></span>"
+    #fmt    = "<span class=\"hashtag\">%s(%i)</span>"
     for key in d.keys():
-        output.append(fmt % (key, d[key]))
+        link = rell % key[1:]
+        output.append(fmt % (link, key, d[key]))
     return output
 
 class Paginator:
-    def __init__(self, x, y, loc="./pages"):
+    def __init__(self, x, loc="./pages"):
+        if x <= 0:
+            print("Error: No posts (x=%i" % x, file=sys.stderr)
+            raise Exception
         self.TOTAL_POSTS = x
-        self.PPP         = y # posts per page
+        try:
-        self.TOTAL_PAGES = int(x/y)
+            setting = "postsperpage.txt"
-        if self.TOTAL_PAGES > 0:
+            if os.path.exists(setting):
-            self.SUBDIR = loc
+                with open(setting, 'r') as f:
-            if not os.path.exists(loc):
+                    self.PPP = int(f.read())
-                os.makedirs(loc)
+            else:
-            self.FILENAME = "%i.html"
+                with open(setting, 'w') as f:
-        pass
+                    f.write(20)
+        except:
+            self.PPP = 20
+        self.TOTAL_PAGES = int(x/self.PPP)
+        self.SUBDIR      = loc
+        if not os.path.exists(loc):
+            os.makedirs(loc)
+        self.FILENAME = "%i.html"
+        try:
+            with open ("lastfullpage.txt", 'r') as f:
+                self.lastfullpage = int(f.read())
+        except : #possible exceptions FileNotFoundError, ValueError
+            self.lastfullpage = 0
+        self.written = []
 
     def toc(self, current_page=None, path=None): #style 1
         if self.TOTAL_PAGES < 1:
@@ -155,24 +185,36 @@ class Paginator:
                 anchors.append("<b>[%i]</b>" % i)
         return "\n".join(anchors)
 
-    def singlepage(self, template, tagcloud, timeline, i=None, p=None):
+    # makes one page
+    def singlepage(self, template, tagcloud, timeline_, i=None, p=None):
         tc  = "\n".join(tagcloud)
-        tl  = "\n\n".join(timeline)
+        tl  = "\n\n".join(timeline_)
-        tbl = self.toc(i, p)
+        toc = self.toc(i, p)
-        return (template % (self.TOTAL_POSTS, tc, tbl, tl))
+        return template.format(
+            postcount=self.TOTAL_POSTS, tags=tc, pages=toc, timeline=tl
+        )
 
-    def paginate(self, template, tagcloud, timeline):
+    def paginate(self, template, tagcloud, timeline, override=False):
+    # override boolean currently reprsents whether or not 
+    # it is a main timeline or a tagline being paginated
+    ## effort-saving feature does not work for taglines currently
         outfile = "%s/%s" % (self.SUBDIR, self.FILENAME)
+        #print(outfile, file=sys.stderr)
         timeline.reverse() # reorder from oldest to newest
-        for i in range(self.TOTAL_PAGES):
+        start = 0 if override else self.lastfullpage
-            with open(outfile % i, 'w') as fout:
+        for i in range(start, self.TOTAL_PAGES):
+            fn = outfile % i
+            with open(fn, 'w') as f:
+                self.written.append(fn)
                 prev = self.PPP * i
                 curr = self.PPP * (i+1)
                 sliced = timeline[prev:curr]
                 sliced.reverse()
-                fout.write(
+                f.write(self.singlepage(template, tagcloud, sliced, i, "."))
-                    self.singlepage(template, tagcloud, sliced, i, "."))
+        if not override:
-        pass
+            with open("lastfullpage.txt", 'w') as f:
+                f.write(str(self.TOTAL_PAGES))
+        return
 
 if __name__ == "__main__":
     def get_args():
@@ -181,7 +223,7 @@ if __name__ == "__main__":
             msg = '''This is microblog.py. (%s/3 arguments given; 4 maximum)
 \tpython microblog.py [template] [content] [optional: email]
 '''
-            print(msg % argc)
+            print(msg % argc, file=sys.stderr)
             exit()
         # script = argv[0]
         template = sys.argv[1]
@@ -189,34 +231,74 @@ if __name__ == "__main__":
         email    = sys.argv[3] if (argc >= 4) else None
         return template, content, email
 
-    # change the path of css for pages in a separate dir from index
+    # assume relative path
-    def adjust_css(template):
+    def adjust_css(template, level=1):
-        # in order
+        prepend = ""
-        x = template.find("./style.css")
+        if level == 1:
-        y = template.find("./timeline.css")
+            prepend = '.'
-        chars = [char for char in template]
+        else:
-        chars.insert(x, '.')
+            for i in range(level):
-        y += 1
+                prepend = ("../%s" % prepend)
-        chars.insert(y, '.')
+        css1 = "./style.css"
-        return "".join(chars)
+        css2 = "./timeline.css"
+        #sys.stderr.write(("%s%s" % (prepend, css1) ))
+        tpl = template.replace(css1, ("%s%s" % (prepend, css1) ))
+        return tpl.replace(css2, ("%s%s" % (prepend, css2) ))
 
-    def main():
+# needs review / clean-up
-        template, content, email = get_args()
+# ideally relate 'lvl' with sub dir instead of hardcoding
-        tl, tc = make_timeline(content, email)
+    def writepage(template, timeline, tagcloud, subdir = None):
-        tcl    = make_tagcloud(tc)
+        html   = ""
-        count  = len(tl)
-        html = ""
         with open(template,'r') as f:
             html = f.read()
-            #print(html % (count, "\n".join(tcl), "\n\n".join(tl)))
+        count  = len(timeline)
-        count = len(tl)
+        try:
-        posts_per_page = 5
+            if subdir == None:
-        pagectrl = Paginator(count, posts_per_page)
+                pagectrl = Paginator(count)
-        if count <= posts_per_page:
+            else:
-            print(pagectrl.singlepage(html, tcl, tl))
+                pagectrl = Paginator(count, subdir)
-        else:
+                if not os.path.exists(subdir):
-            latest   = tl[:posts_per_page]
+                    os.mkdir(subdir)
-            print(pagectrl.singlepage(html, tcl, latest))
+        except:
-            pagectrl.paginate(adjust_css(html), tcl, tl)
+            print("Error: value <= 0 submitted to paginator constructor", 
+                  file=sys.stderr)
+            exit()
+        latest = timeline if count <= pagectrl.PPP else timeline[:pagectrl.PPP]
+        lvl    = 1
+        if subdir == None: # if top level page
+            ovr    = False
+            tcloud = make_tagcloud(tagcloud, "./tags/%s/latest.html")
+            print(pagectrl.singlepage(html, tcloud, latest))
+            tcloud = make_tagcloud(tagcloud, "../tags/%s/latest.html")
+            pagectrl.paginate(adjust_css(html, lvl), tcloud, timeline, ovr)
+        else: # if timelines per tag
+            ovr = True
+            lvl = 2
+            tcloud = make_tagcloud(tagcloud, "../%s/latest.html")
+            fn = "%s/latest.html" % subdir
+            with open(fn, 'w') as f:
+                pagectrl.written.append(fn)
+                f.write(
+                    pagectrl.singlepage(
+                        adjust_css(html,lvl), tcloud, latest, p="."))
+                pagectrl.paginate(
+                    adjust_css(html, lvl), tcloud, timeline, ovr)
+        return pagectrl.written
 
+    def main():
+        tpl, content, email = get_args()
+        tl, tc, tg = get_posts(content, email)
+        # main timeline
+        updated = []
+        updated += writepage(tpl, tl, tc)
+        # timeline per tag
+        for key in tg.keys():
+            tagline = []
+            for index in tg[key]:
+                tagline.append(tl[index])
+            # [1:] means to omit hashtag from dir name
+            updated += writepage(tpl, tagline, tc, "./tags/%s" % key[1:]) 
+        with open("updatedfiles.txt", 'w') as f:
+            for filename in updated:
+                print(filename, file=f) # sys.stderr)
     main()

neouploader.py

@@ -0,0 +1,53 @@
+
+import sys, subprocess, getpass, pycurl, urllib.parse
+if __name__ == "__main__":
+    def api_upload(endpoint, dest_fmt = "/microblog%s%s"):
+        pages = []
+        with open("updatedfiles.txt") as f:
+            pages = f.readlines()
+        c = pycurl.Curl()
+        c.setopt(c.URL, endpoint)
+        c.setopt(c.POST, 1)
+        for page in pages:
+            p = page.strip('\n')
+            i = p.rfind('/')
+            # folder = p[1:i]
+            # file   = p[i]
+            destination = dest_fmt % (p[1:i], p[i:])
+            source = p[2:] # omit './'
+            print("sending @%s to %s" % (source, destination))
+            exists = True
+            try:
+                with open(source, 'r') as f:
+                    pass
+            except FileNotFoundError as e:
+                exists = False
+                print(e)
+            if (exists):
+                c.setopt(c.HTTPPOST, [(destination, (c.FORM_FILE, source))])
+                try:
+                    c.perform()
+                except pycurl.error as e:
+                    print(e)
+        c.close()
+
+    def main():
+        if len(sys.argv) < 2:
+            print("Usage: neouploader.py [neocities username]")
+            return
+        try:
+            pw = getpass.getpass(prompt="Password: ")
+        except KeyboardInterrupt:
+            print("Aborted.")
+            return
+        finally:
+            if len(pw) == 0:
+                print("Empty input. Exiting.")
+                return
+        p = urllib.parse.quote(pw, safe='')
+        target = "https://%s:%s@neocities.org/api/upload" % (sys.argv[1], p)
+        del pw
+        del p
+        api_upload(target)
+        del target
+    main()