By using this site, you agree to have cookies stored on your device, strictly for functional purposes, such as storing your session and preferences.

Dismiss

Fix markdown list indentation parsing and nested lists

roundabout,
created on Sunday, 31 March 2024, 13:05:33 (1711890333), received on Wednesday, 31 July 2024, 06:54:43 (1722408883)
Author identity: vlad <vlad.muntoiu@gmail.com>

0569742706e40107435b6db414e2afa840831c4e

markdown.py

@@ -326,7 +326,7 @@ def tokenise(source):

                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                            inner_content = lines[i][2:].strip() + "\n"      # discard marker and space
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                            i += 1
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                            while i < len(lines) and lines[i].strip() and not ((lines[i].startswith("*") or lines[i].startswith("+") or lines[i].startswith("-")) and lines[i][1:].startswith(" ")):
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                                            inner_content += lines[i] + "\n"
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                                            inner_content += lines[i][1:] + "\n"
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                                i += 1
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                            
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                            content.append(ListItem(inner_content))
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        

@@ -404,6 +404,7 @@ def tokenise(source):

                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                               and not lines[i].startswith("- ")
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                               and not lines[i].startswith("~~~")
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                               and not lines[i].startswith("```")
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                                           and not re.match(r"^\d+\.", lines[i])
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                               and lines[i].strip()):
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                            content += lines[i].strip() + "\n"
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                            i += 1
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        

@@ -448,32 +449,65 @@ if __name__ == '__main__':

                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                # Generate an AST from a markdown file
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                ast = tokenise(
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                        r"""
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        Hello World
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        ===========
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        Lorem
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        ipsum
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        dolor
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        sit
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        amet.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        1. Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        2. Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        3. Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * Lorem
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          ipsum
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        > Hello
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        > World
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        > * Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        > * Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        > * Test
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        Changelog for version 0.1.0
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        ===========================
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        This is the first non-patch update to the project, still in άλφα state.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        New features in this release
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        ----------------------------
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * Give each user a `.config` repo, which is used for storing settings, like
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          userstyles.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Userstyles can be added to the `.config` repo under the name `theme.css`,
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            and they will be applied to the UI.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Efficient UI CSS variables can be overridden as well.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * The `.config` repo is created automatically when a user is created.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * The `.config` repo is private by default, but can be made public or
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            unlisted like any other.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Right now, the `.config` repo is not used for anything else, but it will
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            be in the future.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * When repo deletion will be added, the `.config` will not be deletable.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Currently, it is not possible for previous users to get a `.config` repo,
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            but this will be added very soon.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * Render markdown files, like this one, on file pages.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Markdown comments are not yet rendered: that requires caching so loading
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            comments is not too slow.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Markdown is rendered using a custom renderer, which is _not_ CommonMark
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            compliant.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Some features are missing, but it supports extra features like ++diffs++ or
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            ~~strikethrough~~, and even *******7-star emphasis*******.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            * Missing features include:
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Tables
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Footnotes
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Definition lists
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Task lists
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Indented code blocks
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Ordered lists (will be fixed with patch)
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * ID attributes
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Subscript and superscript
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Setext headers
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * URL title attributes
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * Backslash escaping for format specifiers
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                              * HTML entities
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Old features like trailing space hard breaks are not supported.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * No subset of HTML is rendered yet.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        Changed features in this release
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        --------------------------------
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * Make the tab component styling look nicer.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Colour now matches the nav tabs.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        Fixed bugs in this release
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        --------------------------
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * The `safari-pinned-tab.svg` file now matches the updated logo.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * Before it matched a really similar development logo.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                        * When accessing a nonexistent repository from the web interface, it now returns
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          a 404 error instead of a 403 error.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                          * This was because the access control was done before the repository existence
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            check.
                                        
                                        
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                            """
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                )
                                        
                                        
                                            
                                            
                                            
                                            
                                        
                                    
                                
                                
                                
                            
                                
                                    
                                        
                                            
                                                # for i in ast: