The increasing penetration level of distributed energy resources (DERs) increases the risk of congestion in the distribution network. To mitigate this, the concept of the small-scale DER aggregator was introduced as a change from uncoordinated to coordinated DERs. However, without appropriate network use cost allocation, the unwanted DER curtailment will be enforced by the network operator. Therefore, this paper proposes a new approach for congestion management by allocating the different network usage costs depending on how much congestion is caused by the DERs in the distribution network. For this, a modified Kirschen’s tracing method is proposed and applied to the small-scale DER aggregator market. To verify the effectiveness of the proposed method, a simulation of the small-scale DER aggregator market in South Korea was performed under the IEEE 69-bus distribution network. The model was able to allocate the different network usage costs at different buses and, thus, encouraged the DERs to reduce their generation by charging the energy storage system (ESS) to mitigate congestion. An economic benefit analysis was also performed from the point of view of the aggregator concerning whether they should have an ESS or not.