Social Image Analysis Paper

Personalized travel sequence recommendation is from both travelogues and community contributed photos and the heterogeneous meta-data (e.g., tags, geo-location, and date taken) associated with these images[1]. Availability of cheap location sensor for geo-tagging of images on social media has made geo-tagged images very popular. The approaches which are existing usually mine geographical characteristics using a subset of multiple types of image contents or combining those contents linearly, which ignore correlations between different types of contents, and their geographical distributions.
In this paper we studied a novel method to discover geographical characteristics of geo-tagged social images using a geographical topic model called geographical topic model of social images (GTMSIs)[2].This model integrates multiple types of social image contents also the geographical distributions, in which image topics are modeled based
on both vocabulary and visual features. After survey we came across the the following situations that although there exit many successful text-based data mining approaches or vision-based approaches of analyzing geo-tagged images ,they were not able to address following problems:
1) Geo-tagged social image contains multiple forms of content.
It is very common that text, visual content, and GPS record exist simultaneously on the same social image. Incorporating this rich information may potentially help us to discover the latent information to capture the geographical characteristics of image content. However, this pursuit is nontrivial. It needs to incorporate different type of contents simultaneously using a multi modal model.
2) Visual content and textual description are correlated with each other, and the correlation is different across different regions. Thus, it is reasonable to use middle-level feature, i.e., topic, to capture the correlation between visual content and textual description and model the geographical distribution of the correlation.
3) In reality, there are also many images that are not geo-tagged or do not have any tags. Thus, it needs to analyze these multiple types of image contents and their correlation to support these applications, such as image location prediction and automatic image
tagging.

In this paper, we studied a generative model geographical topic model of social images (GTMSIs) for geo-tagged image pattern modeling, by simultaneously incorporating geographical information, textual description, and visual contents. Previously, many works used the relation between image visual contents and locations to predict location directly. There also exist feature-based geometric matching approaches, applying to co-register online famous landmark photographs for summarization and browsing . The Geo-informative attributes are obtained for each locations based on image visual contents . A soft bag-of-words method is proposed for mobile landmark recognition based on discriminative learning of image patches . As social images usually contain multiple types of contents that are correlated to geographical locations. Those works are limited to visual content analysis. On other side, many works use probability model to discover topics from the text content of geo-tagged images. Latent geographical topic analysis (LGTA)[2] combines geographical clustering and topic modeling to identify the geographical topics of social images, as well as estimate the topic distributions in different geographical locations for topic comparison. Theres another work proposes a language model based on user annotations, to place the annotated Flickr images on the map . The multi-Dirichlet process (MDP)[2]-based geographical topic model captures dependencies between geographical regions to support the detection of text topics with complex, non-Gaussian distributed spatial structures . The model is based on an MDP. Those works use a pure language model to identify the geographical topic distributions of image, which do not combine visual contents in geographical topic model to identify the visual patterns of topics as well as their distributions over regions. GeoFolk[2] uses a generative model to combine text and spatial information together, in which each topic generates latitude and longitude from two topic-specific Gaussian distributions.However, these models and approaches are also used to discover geographical topics from geo-tagged text documents. They cannot be applied to geo-tagged social images directly. The GTMSI model can be widely applied to geo-based clustering, geo-based image retrieval, geographical event detection, point of interest recommendation, etc., giving wide scope for future development.