debugger fails if source code is unavailable

Under specific conditions the code at tbtools line 428 raises an error during traceback rendering. This causes the browser to show a fairly meaning less message

`Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.`
This was with Flask==3.1.1.

In my particular case the original error as shown in the terminal was
File "<string>", line 25 lxml.etree.XMLSyntaxError: Opening and ending tag mismatch: p line 24 and story, line 25, column 10

follow the tutorial at https://docs.reportlab.com/demos/flask_reportlab/flask_reportlab/
but in the preppy template line 24 replace <p>{{ say }}>/p> with <p>{{ say }}/<p>
this causes a parse error and the traceback render fails because line_idx is a bad index.

The terminal traceback continues
`During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/robin/tmp/rlflask/.py313/lib/python3.13/site-packages/werkzeug/serving.py", line 370, in run_wsgi
execute(self.server.app)
...........
File "/home/robin/tmp/rlflask/.py313/lib/python3.13/site-packages/werkzeug/debug/tbtools.py", line 428, in render_html
File "/home/robin/tmp/rlflask/.py313/lib/python3.13/site-packages/werkzeug/debug/tbtools.py", line 428, in render_html
render_line(lines[line_idx], "current")
~~~~~^^^^^^^^^^
IndexError: list index out of range`

The traceback should have been rendered as expected

Environment: x86_64 linux 6.14.6

Python version: 3.13.3
Werkzeug version: 3.1.3

If I change the failing line in tbtools.py with if 0<=line_idx<len(lines): render_line(lines[line_idx], "current") then I do get a rendered traceback. The console cannot be entered at the lxml frame, but that is probably expected.

Please reformat this to show the entire traceback as a code block. Also, please include a minimal reproducible example with steps to reproduce the issue.

The minimal example

from flask import Flask
app = Flask(__name__)

def rmlfail():
	from rlextra.rml2pdf.rml2pdf import go, _invalid_rml_template
	import io
	buf = io.BytesIO()
	rml = _invalid_rml_template % ('','<para>A/<para>')
	go(rml,buf)
	return 'SUCCEEDED'

@app.route("/")
def hello_world():
	return "<p>Hello, World!</p>\n<pre>%s</pre>" % rmlfail()

The terminal output

(.py313) robin@minikat:~/devel/tflask
$ flask --app app run --debug
 * Serving Flask app 'app'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 125-191-719
127.0.0.1 - - [20/May/2025 16:29:03] "GET / HTTP/1.1" 500 -
Error on request:
Traceback (most recent call last):
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/werkzeug/debug/__init__.py", line 343, in debug_application
    app_iter = self.app(environ, start_response)
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/flask/app.py", line 1536, in __call__
    return self.wsgi_app(environ, start_response)
           ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/flask/app.py", line 1514, in wsgi_app
    response = self.handle_exception(e)
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/home/robin/devel/tflask/app.py", line 15, in hello_world
    return "<p>Hello, World!</p>\n<pre>%s</pre>" % rmlfail()
                                                                            ~~~~^^
  File "/home/robin/devel/tflask/app.py", line 10, in rmlfail
    go(rml,buf)
    ~~^^^^^^^^^
  File "rlextra/rml2pdf/rml2pdf.py", line 7527, in go
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/rlextra/radxml/xml2tt.py", line 44, in __call__
    return LxmlTT(**kwds).ttFromSrc(src)
           ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/rlextra/radxml/lxmltt.py", line 63, in ttFromSrc
    return self.ttFromFile((io.BytesIO if isinstance(s,bytes) else io.StringIO)(s))
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/rlextra/radxml/lxmltt.py", line 46, in ttFromFile
    tree = ET.parse(f,self.parser)
  File "src/lxml/etree.pyx", line 3590, in lxml.etree.parse
  File "src/lxml/parser.pxi", line 1975, in lxml.etree._parseDocument
  File "src/lxml/parser.pxi", line 1995, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1875, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1105, in lxml.etree._BaseParser._parseUnicodeDoc
  File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 743, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 672, in lxml.etree._raiseParseError
  File "<string>", line 15
lxml.etree.XMLSyntaxError: Opening and ending tag mismatch: para line 14 and story, line 15, column 9

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/werkzeug/serving.py", line 370, in run_wsgi
    execute(self.server.app)
    ~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/werkzeug/serving.py", line 333, in execute
    for data in application_iter:
                ^^^^^^^^^^^^^^^^
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/werkzeug/debug/__init__.py", line 358, in debug_application
    html = tb.render_debugger_html(
    
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/werkzeug/debug/tbtools.py", line 346, in render_debugger_html
    "summary": self.render_traceback_html(include_title=False),
               ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/werkzeug/debug/tbtools.py", line 295, in render_traceback_html
    row_parts.append(f"<li{info}>{frame.render_html(mark_library)}")
                                  ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/robin/devel/tflask/.py313/lib/python3.13/site-packages/werkzeug/debug/tbtools.py", line 428, in render_html
    render_line(lines[line_idx], "current")
                ~~~~~^^^^^^^^^^
IndexError: list index out of range

Steps to replicate

mkdir tflask && cd tflask && create app.py
python313 -mvenv .py313 && source .py313/bin/activate
wget https://www.reportlab.com/ftp/rlCgi-2.6.1-py3-none-any.whl
wget https://www.reportlab.com/ftp/rlextra-4.4.1-py3-none-any.whl
pip install flask rlCgi-2.6.1-py3-none-any.whl rlextra-4.4.1-py3-none-any.whl
flask --app app run --debug
visit http://127.0.0.1:5000

Adding a breakpoint at the point of failure reveals the probable cause of this issue

self.filename=='rml2pdf.py', self.lineno==7527, len(lines)==17

the module rlextra.rml2pdf.rml2pdf is obfuscated and is just a placeholder that imports and overwrites itself with a compiled version (in this case rml2pdf_313c.py). This is purely for commercial reasons ie to avoid giving away the source code.

The bug/problem here is that there appears to be an assumption that source is available. That is not always the case.

I'm not sure exactly how one can distinguish the case when a malevolent programmer hides / obfuscates the source code because the boss doesn't want to release it. If the bug had occurred at line 10 of the obfuscated code then presumably tbtools would have been happy to report the bug as happening at line 10 of the trampoline code.

If I put in the fix I suggested ie if 0<=line_idx<len(lines): render_line(lines[line_idx], "current") then the traceback is happy to be rendered and the failing line is shown as File "rlextra/rml2pdf/rml2pdf.py", line 7527, in go.

Navigation Menu

Repository navigation

debugger fails if source code is unavailable #3044

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

debugger fails if source code is unavailable #3044

Description

Activity

davidism commented on May 20, 2025

replabrobin commented on May 20, 2025

replabrobin commented on May 21, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions