..for me the only benefit HDMI has over display port is the ability to carry audio, which is great for hooking stuff up to a TV but not amazing for anything else.
Both of them can carry audio.
DisplayPort has some PC features that HDMI lacks (such as daisy-chaining multiple displays, and auxiliary channels that can be used for USB, Ethernet, or basically anything you want) while HDMI has some home theater features that DisplayPort doesn't have, like audio return channel and CEC (although it would be possible to easily support these through DisplayPort's aux channels).
They both support 8-channel digital audio (Although HDMI 2.0 will increase that)
They both support 16bpp color depth
They both support 3D
They both support Ethernet
DisplayPort can natively output to both HDMI and DVI, while HDMI can only adapt natively to DVI.
The current revision of DisplayPort has more bandwidth than both the current and future versions of HDMI (21.6Gbps vs 10.2 soon 18.0Gbps on HDMI). A newer revision of DisplayPort is coming in the near future which will increase DP's lead to 32.4Gbps. Currently, only DisplayPort can do 4K at 60Hz. HDMI is limited to 4K at 24Hz. HDMI 2.0 will increase that to 4K at 60Hz, but the also upcoming DisplayPort 1.3 will up the ante to 4K at 120Hz.
G-Sync and FreeSync are only supported over DisplayPort.
HDMI charges a royalty fee for implementation (per device, plus $10,000 a year base). DisplayPort is royalty free.
And finally DisplayPort has a locking connector design which people seem to like better.