It is traditional at academic conferences to wear clothes of some sort. Exactly what kind of clothes varies tremendously. Some dress up rather business like while others are pretty casual. Perhaps “business casual” would work best (says the guy who once gave a paper in Europe while wearing denim shorts and a T-shirt). In any case, don’t sweat it.