Display screen or portion thereof with icon set

[0001]The FIGURE is a front view of a display screen or portion thereof with icon set showing the claimed design.

[0002]The peripheral dashed broken lines depict a display screen or portion thereof, and form no part of the claimed design. The solid line brackets indicate an icon set. The elements of the set are arranged relatively on the drawing sheet for the convenience of illustration, and the placement of each element relative to another, or to the drawing sheet, form no part of the claimed design.