Category: Development

Fantastic time presenting at Laracon AU 2019

Laracon AU just wrapped up in Sydney. It was a fantastic 2 days of learning (and 2 nights of partying).

I had a blast, doing my first ever presentation at a conference – big shout out to Michael Dyrynda for giving me an opportunity to be a speaker this year. If you are interested in watching my presentation, I am informed that they will all be up on the LaraconAU YouTube by the end of the year (after processing & editing)

https://www.youtube.com/channel/UCYGmktKAAJWnqGGaH9DcTqw

You can find the slides here, though most of my presentation was a demo (which went off without a hitch!)

https://docs.google.com/presentation/d/1ToZNm9L2yIBtpKafRh69O7x5ZA0puPAtMCLeXPdYeAg/edit?usp=sharing

Everyone was great and supportive, many people came up to me to tell me they found my presentation (on my API boilerplate) really interesting, and were keen to have a go. You can find the github for it here –

https://github.com/specialtactics/laravel-api-boilerplate

Me doing my presentation, bit of a postman demo

Me with some of my friends and colleagues

Advanced testing of Laravel packages

I’ve done a talk at the Sydney Laravel meetup today about techniques for advanced testing of Laravel composer packages.

Essentially I take viewers through integration testing, and how to test against a realistic fully-fledged Laravel application in order to have more comprehensive and integration level tests for their Laravel packages.

This is really useful for any package developers, or even just contributors. Although the topic is slightly complex, it’s distilled fairly clearly, and the end result of what to do is actually quite simple!

Along with the usual slides, below you will find one of my Laravel packages, which I used during the presentation to demonstrate the concepts in the talk.

GitHub repo used for examples

Slides

Laravel API Boilerplate

Over the last few months I’ve been working on a very practical github project – a boilerplate for laravel, useful for API projects. The idea here is as I say, to re-image laravel as a framework built for developing APIs.

The main goal is to make API development significantly quicker and easier, and dramatically minimise the amount of code required to get going, in building API components.

In the last few months I’ve done presentations on it at Sydney’s Laravel and PHP meetups, and it’s just starting to get some traction. If you work with APIs, or would like to – I would encourage you to check it out. It is very easy to get started with – in fact that is on of the goals 🙂

Github: https://github.com/specialtactics/laravel5-api-boilerplate/

Slides Link

My Laravel meetup presentation – modern components in old codebases

On Tuesday, me and a colleague are going to do a presentation at the Sydney laravel meetup, where we will talk about our learnings of embedding a multitude of laravel (and other modern) components in an old codebase.

This is very useful for anyone either working on older projects or maintaining an old project, as you can take and use some of the best features of Laravel even in the worst codebases. If you’re anything like me, the concept of not having laravel is a scary one, so come along and learn from our experiences!

Meetup event

Update:
You can see the slides for the presentation here.

Working with Beanstalk Queues

Queueing is a natural part of modern web applications. Without it, you will end up with a poor user experience due to unnecessary execution blocking leading to inadequate response times. In Laravel (which everyone is using, of course), one of the easiest providers to get going with queues is beanstalkd. You can set it up in 15 seconds on any linux distro – and away you go without any configuration necessary. However when you need to debug and investigate, it isn’t really clear how to proceed. As such, I wanted to quickly go over some tools that I’ve discovered which are very helpful.

CLI based tube inspection

If all you have is a CLI, and you want a hassle-free way to check your tubes, I highly recommend beanstool. With no setup necessary, it will quickly give you all the info you need.

Web based tube inspector

If you need a GUI and/or more detail, then beanstalk console has you covered. It’s a very simple and lightweight PHP application which will give you plenty of info, and allow you to quickly and efficiently go through your tubes if they are clogged with several jobs.

Clear a queue in Laravel

This is perhaps the single most useful one. The scenario is that you have hundreds or thousands of faulty jobs stuck in your queue, and you want to quickly clear all of them. Amazingly, there is no trivial out-of-the-box way to do that, so I’ve come up with the following Laravel Command to get that job done.

The Value of Learning

Today at work, at our Monday general meeting, I gave a brief speech about my take on our value of Learning… so why not also share it with the world?

I believe that no other industry other than ours is so fast paced in terms of the technology needed to accomplish our jobs. Something which might be relevant one year, will be shunned and largely disused the next. Every week there is a new acronym for a new developer tool, every month there is an update to a framework we use. Development paradigms are always changing.

In short, if we are to excell above our competitors, we must truly embrace all facets of the learning process. We quite simply don’t have the right to call ourselves professionals without fully exhausting all the avenues we have at our disposal to learn new things.

These will come in the form of online seminars, meetup groups and conferences, discussing projects with colleages, work outside of work, always helping people – on stack overflow, forums and IRC, hours and hours of reading online documentation at midnight, and a good old fashioned book or two.

The media and news programs always report that our profession has benefits and culture which is the envy of every worker on the planet. I for one believe we should work hard to earn this place of ours in the world.

The Laravangular Stack (with Postgres)

Let’s start March with some tech talk.

I’m going to be talking about technologies that we use to develop new web applications from scratch. For normal applications (ie. the 98% thereof which contrary to what hipster devs on your team believe, do not have a legitimate use case for mongo), this is the technology stack which I feel is far superior to any other choice. I call it the Laravangular Stack.

To begin with, all modern web applications must be designed API first with a separate backend and frontend. The reasons for this are obvious;

  • Separation of concerns
  • Easier automated testing
  • Code once for multiple consumers (ie. web, mobile, device, API for 3rd parties, etc)
  • Performance
  • Easier to upgrade components
  • Easier to debug
  • Easier to divide dev workload
  • Your application architecture isn’t a sack of shit
  • Ability to have a rich, interactive frontend (and stop pretending that jQuery can accomplish this)

The question then becomes, what specific technologies do we use for such an application? In my view these answers are obvious.

Laravel has you covered on the backend
Laravel is the best thing that’s ever happened to web programming period. The most popular framework has you totally covered for API-first projects, and will deliver more features with more simplicity than anything else in existence – while the community is hard at work churning out thousands of packages for you to leverage in your project. It’s an obvious choice.

Angular for a superb user experience
Angular, at least for now, is the only proper actual front-end framework. Yes, it’s far from perfect, and some have even gone so far as to call it a tech demo…. however until Angular 2 comes out, there isn’t anything which can compete with it because everything else is essentially a bit of wet lettuce. Frontend JS frameworks are a very recent invention, and it’s an interesting new space – but the sooner you come to terms with the fact that Angular at the moment simply shits on everything else out there, the sooner you can begin to create world-beating applications.

PostgreSQL for RDMS
Many people still use MariaDB (or god forbid MySQL), however I’ve felt for a long time that it is rather inferior to PostgreSQL. A project I’m working on now has cemented my view, with 2 rather sharp examples.

Our first problem stemmed from having a very high write workload for one logging aspect in our system. The use case involves a very large number of external devices writing logs to our system every few minutes. In our testing, InnoDB is a severe bottleneck for write-heavy workloads. I have read up on this, and the tl;dr is that there is no way to fix it – it’s just the way it’s designed. That rather sucks.
The second problem are UUIDs. UUIDs are pretty much the gold standard now, especially when it comes to anything API. How do you store them in a database though? Well, in the case of MariaDB, there is no perfect solution. If you want optimal performance, you can choose to store it as a binary 16 – and say goodbye to ever being able to manually read or write any UUID. Alternatively you could store it as varchar 36 (or 32 without the dashes), but take a non-trivial performance hit.

PostgreSQL on the other hand has had a naitive UUID field for a while now, which is internally stored as if it were binary, but behaves like a char. Likewise, it has no performance problems with write-heavy workloads. Perfect!

Of course when you come down to the bottom of it, there are many more differences between MariaDB and PostgreSQL – and you’ll likely find that PostgreSQL can do everything MariaDB does, but also more. It just so happens that PostgreSQL is always first to the party with any number of features that it’s had first in the last decade or more, including being an actual database system with full ACID compliance. It’s only pitfall if any is the lack of tools (relatively speaking) compared to MariaDB. If you can get past that barrier however, I would definitely recommend it.

Everything else is irrelevant
Whether you use Apache or Nginx, they both proxy pass PHP requests straight to php-fpm in all modern configurations, and let’s face it, nobody serves static assets anymore – it all comes from CDNs. Whatever other technologies used, wherever you are hosted – it’s less relevant than getting the fundamentals right.

Laravel Groupwise Max

For a project I’m working on, I really needed to do a groupwise max in Laravel. If you’re here, you likely don’t need an explanation of what that is, so much as how to actually bloody do it, so without further delay;

Let’s say that we have Actions, and we have an ActionLog, and you want to get the latest ActionLog entry for a series of Actions – determined by their “timestamp” field. You would write this:

Unfortunately there’s no way to do it without the dreaded DB::raw, but otherwise it turns out to be pretty simple.

Missing Auth header in Apache

For some reason Apache 2.4 (and maybe earlier versions, but nobody should care about them) drop the Auth header. No idea why, but here’s the solution:

PostgreSQL vs MariaDB – a recent experience

In developing API-centric backend applications, there are two main points of contention which make me want to use PostgreSQL over MariaDB (or MySQL) right now.

Native Support for the UUID data type
Why is this important? Well, every backend system should only make it’s users (be it an Angular frontend or a mobile app or a 3rd party consumer) aware of the UUID of a given object. Internal incrementing IDs are not the business of the consumer of the API, for all sorts of security and privacy reasons. Plus, dealing with UUIDs allows API consumers to create related objects themselves including their UUIDs and send them straight to the API all together.

To that end, MariaDB simply has no nice way of dealing with them. Performance wise, the best thing to do is to make them a BINARY 16 length field. However, this will make them unreadable in any sort of interface, and certainly not editable – which is simply unacceptable for development and testing. And yes, I am aware it’s possible to filter them down to readable format, but who the hell has the time to do that everywhere and everytime you need to see a UUID? Ridiculous!

That just leaves the option of making them a char 32 length field – but that has a significant performance drag over binary (which is slower still than an integer field already!).

Enter PostgreSQL. It has a naitive UUID field whereby it is internally stored as a binary for maximum performance, but as far as it’s displayed to the user, it will in all situations act like a character field type. This is precisely the sort of functionality developers want in modern applications. I know it’s on the bucket list for MariaDB, but who knows when it will be implemented.

Horrible Performance for mass inserts
InnoDB is a terrible engine – no two ways about it. That’s why MariaDB devs are working furiously on XtraDB, to replace it. Meanwhile, one of the biggest problems users are stuck with is horrendous insert performance.

Let’s say you have an application that is constantly receiving a large number of records, which must all go into a certain table. Sure you can store them in memory as they come in – concurrency in the application itself need not be a problem. But what happens when you have too much data for InnoDB itself to insert in a timely manner? A really big problem is what.

This is another use case where PostgreSQL simply does not suffer from the same performance issues that InnoDB does. And don’t even dare to mention MyISAM, you may as well use floppy disks if you want to go down that route.