How GraphSMOTE works

In the synthetic node generation process, SMOTE method is utilized to amplify the minority class, typically anomalous users. Synthetic nodes are created by replicating existing nodes and their connections, maintaining heterogeneity in links. The "homey" adjacency matrix replaces the original adjacency matrix, enabling the calculation of cosine similarity between adjacent node pairs. This diagram explains GraphSMOTE: In short:

• Generates synthetic nodes for minority class
• Upsampling method “replicates” nodes and connections
• Maintains heterogeneity in links
• Adds minor differences for distinction

How SparseGAD works

Sparsification techniques further refine the graph by filtering unnecessary connections using a threshold δ and limiting connections through KNN. Graph Anomaly Detection (GAD)-oriented regularization is applied to prevent faulty links, enhancing the model's accuracy in distinguishing anomalous users. This diagram explains SparseGAD: In short:

• Fraudulent users tend to link with heterophilic users
• SparseGAD adjusts adjacency matrix to separate homophily and heterophily links
• Filters unnecessary elements in homey adjacency matrix
• Uses threshold delta and KNN to remove unnecessary links

How GNNs works

• GCN aggregates information from neighboring nodes to update node representations, capturing local graph structures effectively.
• GAT employs attention mechanisms to dynamically weigh the importance of neighboring nodes during information aggregation, enhancing the model's ability to capture complex relationships in the graph.
• GraphSage generates node embeddings by sampling and aggregating features from a fixed number of neighbors, enabling scalable and robust representation learning on large and heterogeneous graphs.

Three Main Paths

Our approach to detecting fraud involves three main paths, each of which utilizes a different preprocessing technique before applying GNN models (such as GCN, GAT, or GraphSage). The following visualizes the workflow: In short,

• Baseline Path: Here, we directly input the dataset into the chosen GNN model. This path serves as our baseline for comparison against the other paths.
• GraphSMOTE Path: In this path, we apply the GraphSMOTE technique discussed in the Methods section before feeding it into the chosen GNN model. Again, GraphSMOTE is an oversampling technique, which helps address the imbalance in the dataset by generating synthetic nodes.
• GraphSMOTE with New Adjacency Matrix Path: In this path, we utilize GraphSMOTE and the principles of SparseGAD. Here, the SparseGAD receives the balanced dataset generated by GraphSMOTE and incorporates consine similarity matrix and sparsity constraints which helps highlight significant connections and anomalies in the graph.

Our Findings

As we can see in the table, the GraphSage model performs best on Yelp and Reddit datasets, and the GraphSage with Modified GraphSMOTE performs best on the Amazon dataset.

We first aim to explain why GraphSAGE outperforms GCN and GAT on the Yelp and Reddit datasets, while achieving a similar ROC-AUC score on the Amazon dataset. Our intuition is that this discrepancy arises from the structural differences between the datasets. It's important to note that both the Yelp and Reddit datasets exhibit relatively sparser structures, whereas the Amazon dataset presents a more condensed structure. This condensed structure in the Amazon dataset allows the weights in the GAT model on each neighbor’s features to have greater significance compared to those in the Yelp and Reddit datasets. On the other hand, the GraphSAGE model samples and aggregates from the neighboring nodes. On the Yelp and Reddit datasets, the simplicity of the GraphSAGE method facilitates better model tuning and results in improved convergence, thereby enabling the GraphSAGE model to outperform the GAT model on these datasets.

The main reasons why we believe our GraphSAGE performs well on the Yelp and Reddit datasets, and GraphSAGE with Modified GraphSMOTE on the Amazon dataset, are related to the degrees of connectivity and the number of connected nodes. Below, we present the degree distribution of the datasets.

In the Yelp and Reddit datasets, the median number of degrees is lower (6 and 1 respectively) vs. Amazon (which has 42). This discrepancy may reflect variations in the connectivity and structural complexity of the datasets, which directly influence the performance of fraud detection models. Furthermore, Yelp and Reddit datasets have more nodes with only self-loops (106 and 625 respectively) compared to Amazon (which has 6). This lack of connectivity in the Yelp and Amazon datasets limits the potential for synthetic nodes generated by the SMOTE method to establish meaningful connections within the graph. As a result, the ability of the SMOTE method to effectively balance the class distribution while maintaining dataset structure may be constrained in Yelp and Reddit datasets whereas flourished in the Amazon dataset. Thus, the GraphSMOTE and the Modified GraphSMOTE models were unable to surpass the performance of the baseline GraphSage model

This disparity in performance highlights the importance of tailoring the model to the characteristics of the dataset.