Street-to-shop: cross-scenario clothing retrieval via parts alignment and auxiliary set

We address a cross-scenario clothing retrieval problem- given a daily human photo captured in general environment, e.g., on street, finding similar clothing in online shops, where the photos are captured more professionally and with clean background. There are large discrepancies between daily photo scenario and online shopping scenario. We first propose to alleviate the human pose discrepancy by locating 30 human parts detected by a well trained human detector. Then, founded on part features, we propose a two-step calculation to obtain more reliable one-to-many similarities between the query daily photo and online shopping photos: 1) the within-scenario one-to-many similarities between a query daily photo and an extra auxiliary set are derived by direct sparse reconstruction; 2) by a cross-scenario many-to-many similarity transfer matrix inferred offline from the auxiliary set and the online shopping set, the reliable cross-scenario one-to-many similarities between the query daily photo and all online shopping photos are obtained.