I am using a code block element in Oxygen builder but you can use any code block element or plugin which allows you to write custom PHP.
The code (PHP)
Simply copy and paste the PHP code below into your code block element or plugin (assuming you have the plugin set up correctly to output the query in the correct place).
<?php
$categories = get_categories( array(
'orderby' => 'name',
'order' => 'ASC'
) );
foreach( $categories as $category ) {
echo '<li><a href="' . get_category_link($category->term_id) . '">' . $category->name . '</a></li>';
}
?>
Shoutout to WordPressDave for providing this solution on Stackoverflow, which I then modified.
As you can see, the PHP is quite simple and easy to modify. I wrapped the a-tags with li-tags for my requirements but you can change this as per your requirements. Also, if you need a custom classes, you can easily do that also.