We welcome issue reports and feature requests on GitHub. If you’d like to stay in the loop we have a monthly newsletter and tweet more frequent updates from @pgMustardApp.

Week of Monday 21st October

Core Logic

• Improve allocation of subplan costs: issue #18

Week of Monday 14th October

Core Logic

• Clarify wording when row estimates are accurate to within five percent: issue #17

Usability

• Pinned the timing bar to the top of the screen to provide context while browsing the tree/table

Bugfixes

• Fix an issue where the tree would not be openable if all the nodes summed to less than 5% of total execution time

Account Management

• Stop asking for person’s name on signup (we decided we didn’t really need this information), and ask for permission to send product/content related emails

Week of Monday 7th October

Account Management

• Added GitHub as an authentication option

Week of Monday 30th September

Core Logic

• Added a tip for lossy bitmap scans

• Improved the descriptions of off-but-acceptable row estimates

Account Management

• Allowed people to choose to deny or allow storage of their plans for product improvement purposes. Previously we logged “unusual” plans and ones which caused an exception, but no other ones. Now we will log plans if and only if you tick the box to give us permission. We’ve updated our privacy policy accordingly.

Week of Monday 23rd September

Core Logic

• Added a tip for table bloat

• Added advice in the case where the plan is already very fast

• If pgMustard can’t provide any tips, do not charge a credit for usage

Account Management

• Add a history page to show the last 15 analysed queries

Week of Monday 16th September

Core Logic

• Deal with plans that have extra quotation marks at the beginning and end - some interfaces seem to provide them like this

Week of Monday 9th September

Core Logic

• Adjust the likelihood of the tip recommending an index-only scan to take into account how many columns are required - more columns mean it is less likely to be shown

• Adjust the value of the tip recommending pushing down a filter so that if it only saves time on one operation, and we have a good tip on that operation, it will likely win out

• Rephrase the heading of the tip suggesting restructuring of the query to be more helpful

• Added info about improvements introduced in postgres v12 to CTE scans tip

Bugfixes

• Allow values in the “Operation Detail” section of the reading pane to be wrapped, to prevent long code fragments causing a horizontal scroll bar

Week of Monday 26th August

Core Logic

• A significant rewrite to the way we generate tips. Each potential tip is assigned a score, all tips below a certain threshold are discarded, and at most the best three retained. The best tip will be opened by default. Hopefully this will reduce false positives and give people a better idea of what to try first.

Account Management

• Added a button to cancel paid subscription on the account page. Currently this is only a mailto link which we’ll deal with manually - our paid subscribers (and churn) are small enough that it doesn’t feel worth automating this process end-to-end. We won’t require anything other than a single line email to cancel, and we’ll honour the cancellation on the date the email is received, even if it’s too late to cancel the subscription payment and we have to refund it.

Week of Monday 29th July

Core Logic

• Added a description for GROUP type operations

Week of Monday 22nd July

Account Management

• Added a page where you can manage your account, as well as add/remove people from your subscription

Core Logic

• Move the “unused rows” tip further up the list. After recent changes, this is much less likely to appear as a false positive, and is often the first thing to consider when optimising a query.

Bugfixes

• Fixed an issue which caused submitting queries to fail if they used parallelism, and were generated with the VERBOSE flag but not the BUFFERS flag: issue #12

Week of Monday 15th July

Bugfixes

• Fixed an issue whereby the app might throw an exception under some circumstances where the main thread did no work: issue #8

• Fixed an issue whereby under some circumstances when the main thread was not used, it would still be counted for the purposes of number of threads used by an operation

• Fixed an issue whereby under some circumstances when the main thread was not used, the operation would be described in the UI as not having been run, despite having taken time and produced rows

Week of Monday 8th July

Core Logic

• Add a description for WorkTable scans and Recursive Union operations

• Moved trial from a fixed time period (1 day) to a fixed number of non-example uses (currently 3)

Week of Monday 1st July

Core Logic

• Added a description for foreign scan operation nodes

Usability

• Replaced operation detail and tips popovers with one unified reading pane, which is shown for the highlighted node

• Highlight the longest-running operation which has tips on explore page load

• Prevent highlighting text in tree node and navbar links. These are buttons and it’s just annoying when the browser treats them as text

Bugfixes

• Prevent local storage growing too large by only holding onto the last 15 plan results and deleting the rest - issue #7

Week of Monday 24th June

Core logic

• Removed the generated query. Although we had hoped this would be useful, in UX tests and feedback, we got the message that, although it was cool, it didn’t really help with the process of diagnosing and fixing performance issues, and so we’ve removed it. If we were wrong, get in touch to let us know!

Usability

• Replaced the “send feedback” link on the menu with a link to the issue tracker

Bugfixes

• Fixed an issue which meant timing breakdowns could sometimes be calculated incorrectly for parallel queries which executed more than one loop on a worker thread for an operation - issue #1

• Fixed an issue which meant that certain join queries would be generated in such a way that they could not be successfully rendered - issue #2

• Fixed an issue which caused boolean values not to be displayed on the “more info” operation popover - issue #3

Week of Monday 17th June

Core logic

• Move the tip explaining that a loop-causing operation might be worth investigating from the parent (loop-causing) operation onto the child (looped) operation.

Usability

• Make popovers easier to read by adding more spacing and laying out the key-value pairs as a table, with dividers

• Streamline the existing example plans, adding a short description and adding a new one demonstrating how long query plans are displayed

• Show a message advising use of the BUFFERS and VERBOSE flags when they’re omitted

Bugfixes

• Version css & javascript to make it more likely the browser will load the newest version and you will receive the latest features

• Fix an issue which could cause a failed request when the person generating the query didn’t use the VERBOSE flag

Week of Monday 10th June

Usability

• Allow showing/hiding of tree and table elements

• Automatically hide quick branches of the tree

• Show timing information on tree

• Replace the “i” icon with a magnifying glass in the places where it was used for “more information” (leave it in place where it was used for “what does this mean?”)

• Get rid of duplicate icons on tree nodes with tips and use the lightbulb icon to link to the details

• Move the timing breakdown bar to the top of the page

Bugfixes

• Fix an issue (only reproducible in Firefox on Mac) which meant the text input could appear above the loading overlay

Week of Monday 27th May

Core logic

• Added a description for Merge Append operations

Usability

• Improved rendering of tree - particularly spacing & scaling at different screen sizes

Bugfixes

• Fixed an issue which meant long tree nodes might be truncated or unclickable when viewed in Firefox

Week of Monday 20th May

Core logic

• Updated parsing to accept plans from auto_explain output

• Linked tip on inefficient multi-column indexes to blog post

Usability

• Added in the (optional) AS keyword when generating queries which assign aliases to table names

Week of Monday 13th May

Core logic

• Added a tip that informs you of inefficient index usage (high % of rows removed by filter)

• Improved the tip that points out operation rows are discarded so it doesn’t look further up the tree than the limits of its subquery

Usability

• Include more information (table & index names) that will be useful for resolving issues in tips

Bugfixes

• Fixed an issue that meant the tip wouldn’t show you how many blocks were read by an operation in the case where they were all read from the cache

Week of Monday 6th May

Core logic

• Query generation now supports some subqueries (InitPlan and SubPlan children)

Usability

• Improved advice for using psql and added a modal with more information

• Made the link to the changelog open in a new tab

Bugfixes

• Fixed an issue with vertical modal sizing

Week of Monday 29th April

Core logic

• Query generation now covers joins with a wider range of inner/outer children

• Query generation now covers simple nested loops

Usability

• Slimmed down the nested loop example to make it easier to understand at a glance

• Added a link to get in touch if the query is absent or displayed incorrectly

Week of Monday 22nd April

Core logic

• Include recommendation to try again with a warm cache in the case where the cache hits are low

• Include information on setting server configuration parameters in tips where they are suggested as a possible solution

• Include links to original pgMustard on some anti-tips as well as the corresponding tips

Usability

• Shortcut keys: dismiss popovers on escape and submit form on Ctrl + Enter or ⌘ + Enter

• Change the cursor when hovering over a row in the performance statistics table to try to make it clearer that it can be clicked on to select the operation

Bugfixes

• Fixed an issue which meant popovers with lots of content might overflow the body and be unreadable past a certain point

Week of Monday 15th April

Core Logic

• Added a link to the privacy policy for people to read as they sign up

• Automated paid access for the account when someone pays for full access (previously done manually)

Bugfixes

• Fixed a tip which erroneously recommended VACUUM to improve statistics - it should have read VACUUM ANALYZE.

Week of Monday 8th April

Core Logic

• Added a bar representing time breakdown to the “explore” view

• Improved query generation for WHERE clauses spread over multiple nodes

• Added description for Function Scans

Usability

• Tweaked the size and spacing to make clearer the distinction between different elements in the explore view

Bugfixes

• Fixed an issue that meant that if plans featured a large amount of indentation, tree nodes in the rightmost column could be truncated

• Fixed an issue that meant that if someone navigated directly to a plan link from another website, they might be redirected to the main page

• Tightened up the criteria around detecting ANALYZE data are present, to prevent false negatives in the event of a server error.

• Fixed a bug which meant the overlay underneath a popover could be scrolled off screen

Week of Monday 1st April

Core Logic

• Query & snippet code generation for new operations - Materialize, Bitmap Heap Scans, Bitmap Index Scans, BitmapAnd and BitmapOr operation types.

• Added descriptions for BitmapAnd, BitmapOr and ProjectSet operation types

• Added a link to our new article on writing indexes for tips which recommend adding an index

Bugfixes

• Fixed an issue which meant plans which contained UPDATE or DELETE statements might fail to be processed

• Make sure people get either redirected to the login page or a helpful error message when their session expires

Week of Monday 25th March

Core Logic

• Improved the logic around recommending indexes for slow sequential scans - now amount of rows discarded is considered as a % of total table rows, rather than absolutely, so large sequential scans which discard a relatively small number of rows will no longer receive a tip recommending adding an index.

• Separated out the cases where the table is small and a small number of rows are removed by the filter when explaining why we aren’t recommending an index for a sequential scan

• Included more information in the tips recommending an index on slow sequential scans (the filter being used) and sorts (the sort order being applied) which should give more guidance on what index is needed

• Added a description and tree-level summary for WindowAgg operations

• Show total query time on explore view

• Returned to showing the code snippets as part of the “more information on this node” popover

• Show every key-value pair that we don’t display elsewhere in the “more information on this node” popover

Usability

• Improved messaging and handling of various login failure cases

Bugfixes

• Fixed a bug introduced in the login changes last week which meant people were returned to the http address after login, rather than https

Week of Monday 18th March

Usability

• Streamlined the process of signing up for a trial, so people don’t have to wait for an email before getting started

• Show people trialling the app how long remains of their trial

• Tried to make it clearer which number format we are using - explaining it in the operation time info box, and rounding to two decimal places rather than three (so decimal point is harder to mistake for a thousands separator)

Bugfixes

• Fixed an issue where total number of rows in the table (used when recommending an index on a sequential scan) was calculated incorrectly when the sequential scan was operating in parallel

Week of Monday 11th March

Core logic

• Show a reconstruction of the original query on the exploration view, highlighting the relevant parts when operation nodes are selected. Currently this is only provided for a small proportion of queries.

• Show a summary of the operation for some nodes in the tree - eg the index used, or the initial sort key.

Bugfixes

• Fixed an issue which meant text from tree nodes could not be selected (eg in order to copy)

Week of Monday 4th March

Core logic

• Added new content on index-only scans as a “read more” link

Bugfixes

• Fixed an error which occurred whenever an operation had not been executed at all - ie the number of loops was zero. Also update the way these operations were shown in the UI to make it clear they had not been executed at all.

Week of Monday 25th February

Core logic

• Added a description for the “lock rows” operation

Week of Monday 18th February

Usability

• Replaced popovers with modal dialogues - hopefully these will interfere less with people exploring the data, be easier to read from start to finish and possible to copy text from

Week of Monday 11th February

Core logic

• Added a tip/anti-tip pair to describe whether or not there were trips to the heap on an index-only scan

• Added a tip to recommend an index-only scan on slow index scans

Usability

• Improved rendering of tree view at higher zoom levels/dpi

• Tweaked spacing of buttons on input page on smaller screens

Week of Monday 4th February

Core logic

• Tweaked the logic for when we advise investigating adding an index to speed up a sequential scan, and what information is shown in different circumstances.

Usability

• Improved description of timing to make it clearer that we are showing wall clock time

• Change wording on operations already running quickly to be more clear this is a positive thing!

• Change information given when row estimates are perfect to state that this is the case and give a more concise summary of the data

• Change tips icon to a lightbulb to show this is an opportunity/suggestion to improve 💡

• Change “no tips” icon to a tick to show that this aspect is currently working well

Bugfixes

• Fixed an issue whereby if the rows produced by an operation and the query as a whole were both zero, you could receive a tip saying some the rows produced by the operation were unused

• Fix temperamental link to this changelog

Week of Monday 28th January

Core logic

• Made it clearer when the app is referring to rows per iteration, and when total rows, in the same way as with times. Also improved the “rows discarded” tip to look at total rows rather than rows per iteration

• Support for parallel queries: make sure wall clock time rather than cpu time is displayed (and used for calculations), and show the number of threads used.

• Improve code generation for Aggregate nodes, to specify which functions are being called, eg SUM or MAX

• Added a tip for slow sorts to suggest an ordered index

Usability

• Improved messaging around errors, particularly unexpected ones and session timeouts

• Added a new example that showcases some of the new features

• Reorder the tips so that more specific (hopefully roughly the same as relevant) ones are more likely to appear at the top

• Improved wording of sequential scan tip to make it clearer what the solution is

• Tidied up buttons, making them larger and more evenly sized/spaced

Week of Monday 21st January

Core logic

• Added support for CTE scans, including operation description, code generation and related tips

Usability

• Created changelog and added link from the app

• Change colour scheme to improve readability, increase emphasis of highlighted elements and reduce glare (night mode)

• Replaced example query plan with a more realistic one

Bugfixes

• Fixed an issue whereby cache hits/misses for an operation included those of child operations

• Fixed an issue which caused people trying to navigate to the “explore” results page of the app from a webpage outside the app to be redirected to the initial “input” page of the app

• Fixed a floating-point issue where loops could be presented as taking eg 0.2159999999999ms

• Fixed a bug whereby an operation with lots of loops, each of which took a negligible (< 0.0005ms) time, could be described as having negative loop/total execution time

Week of Monday 14th January

Core logic

• Support for loops: correct operation time calculations, and show time breakdown when an operation is executed more than once (“Actual Loops” > 1)

• If a Nested Loop node causes a node to be executed a lot, consider it a significant operation, and add a tip to point out it contributes more than is obvious to the query time

• Improve the “unused rows” tip to be less easily fooled by aggregating together rows, and specify which node is discarding the rows in the advice

• Show the “bad row count estimate” tip for less incorrect estimates, and include the degree of the error in the advice given

Usability

• Add an indicator (💛) to tree nodes for operations which have a tip

• Added advice on how to get a query plan using psql

• Add a link to allow people to submit feedback via email

• Make the technical information on the operation popover more prominent, and reorder the values to be more readable

• Colour adjustments to improve legibility

• Improve wording of operation descriptions

Bugfixes

• Correct bug that meant the “Index Name” field was sometimes not shown in the operation popover

• Fix a bug that meant that submitting a plan with 0 expected or actual rows would have caused an error

Week of Monday 7th January

Core logic

• Add descriptions for the different read and join operations (eg sequential/index scan, hash join, nested loop etc)

Usability

• Add routing logic, so browser back/forward buttons work between different plan views and the input view

• Add a link to log out

• Stop annoying horizontal jumping when switching between explore/input views by forcing the vertical scroll bar to appear on both pages

Bugfixes

• Fix an issue whereby, if the generated code for an operation was too long, the information popover would flicker and be impossible to view

• Fix an issue which would have allowed cross-site scripting attacks if hosted on localhost

• Fix a UI issue which caused tree node lozenges to be cut off in Firefox

Week of Monday 31st December

Core logic

• In the operation popover, include an example of the code that the node might represent

• Add a description of the operation to the operation popover

Usability

• Correct alignment of “advice” icons (💛) with text

• Show a loading spinny on initial page view, as the javascript is loading.

• Round off the corners joining the lines on the tree

Week of Monday 17th December

Core logic

• Add in a graphical view of the operations in the query plan as a tree

• Add a tip for when the rows produced by an operation do not seem to be used. This replaces the old tip which was shown whenever the number of rows produced by an operation was deemed to be large

• Stop showing the number of loops for an operation, and show the number of rows returned

Usability

• Allow clicking on an operation in the performance table or tree view to highlight it in both views

• Remove the advice on how to obtain a query plan using psql

Week of Monday 10th December

Core logic

• Link to in-depth articles to explain tips in detail

• Show “anti-tips” - aspects of an operation that are functioning well and don’t need any attention. For example, all the data has been read from the cache, or the row count estimate is accurate

• Tweaks to how we calculate which operations are worth focusing on to improve query performance.

• Add a popover for each of the table column headers to explain what the column means

Usability

• Move detailed explanation of tips into a popover to make the tips column less cluttered

• Sort table initially by ID rather than by time, to make it represent the text query plan

• Always sort the table rows in their natural order, depending on the column (eg descending when sorted by times, ascending when sorted by ID), rather than allowing sorting in either direction.

• Add an indicator to show which column the table is currently sorted on

• Refinement of tip wording