gitee/face-search
1
0
Fork 0
mirror of https://gitee.com/open-visual/face-search.git synced 2025-07-10 19:37:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

!8 添加对opensearch的支持，删除对proxima与milvus向量引擎的支持

Browse Source 
Merge pull request !8 from divenswu/dev-opensearch
This commit is contained in:
divenswu 2022-11-23 08:08:55 +00:00 committed by Gitee
commit eefd24b696
No known key found for this signature in database
GPG Key ID: 173E9B9CA92EEF8F
71 changed files with 2510 additions and 3833 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

44
README.md
View File

@ -1,6 +1,6 @@
## 人脸搜索M:N
* 本项目是阿里云视觉智能开放平台的人脸1：N的开源替代，项目中使用的模型均为开源模型，项目支持milvus和proxima向量存储库，并具有较高的自定义能力。
* 本项目是阿里云视觉智能开放平台的人脸1：N的开源替代，项目中使用的模型均为开源模型，项目支持opensearch(1.x版本支持milvus和proxima)向量存储库，并具有较高的自定义能力。
* 项目使用纯Java开发，免去使用Python带来的服务不稳定性。
@ -22,9 +22,7 @@
    2、[onnx](https://github.com/onnx/onnx)
    3、[milvus](https://github.com/milvus-io/milvus/)
    4、[proxima](https://github.com/alibaba/proximabilin)
    3、[opensearch](https://opensearch.org/)
* 深度学习模型
@ -32,15 +30,14 @@
    2、[PCN](https://github.com/Rock-100/FaceKit/tree/master/PCN)
### 版本1.1.0更新
* 1、修复已知BUG
* 2、添加人脸比对1：1接口，详见文档：[05、人脸比对服务](https://gitee.com/open-visual/face-search/blob/v1.2.0/scripts/docs/doc-1.1.0.md#05%E4%BA%BA%E8%84%B8%E6%AF%94%E5%AF%B9%E6%9C%8D%E5%8A%A1)
### 版本2.2.0更新
* 1、添加对opensearch的支持，删除对proxima与milvus向量引擎的支持
* 2、更新：删除搜索结果中的距离指标，仅保留置信度指标（余弦相似度）
### 项目文档
* 在线文档：[文档-1.2.0](https://gitee.com/open-visual/face-search/blob/v1.1.0/scripts/docs/doc-1.1.0.md)
* 在线文档：[文档-2.0.0](scripts/docs/2.0.0.md
* swagger文档：启动项目且开启swagger，访问：host:port/doc.html, 如 http://127.0.0.1:8080/doc.html
@ -51,23 +48,30 @@
<dependency>
<groupId>com.visual.face.search</groupId>
<artifactId>face-search-client</artifactId>
<version>1.1.0</version>
<version>2.0.0</version>
</dependency>
```
* 其他语言依赖
&ensp; &ensp;使用restful接口：[文档-1.2.0](https://gitee.com/open-visual/face-search/blob/v1.2.0/scripts/docs/doc-1.1.0.md)
&ensp; &ensp;使用restful接口：[文档-2.0.0](scripts/docs/2.0.0.md)
### 项目部署
* docker部署，脚本目录：face-search/scripts
```
1、使用milvus作为向量搜索引擎
docker-compose -f docker-compose-milvus.yml --compatibility up -d
1、配置环境变量：FACESEARCH_VOLUME_DIRECTORY，指定当前的挂载根路径，默认为当前路径
2、使用proxima作为向量搜索引擎
docker-compose -f docker-compose-proxima.yml --compatibility up -d
2、对opensearch的挂载目录进行赋权：
新建目录：${FACESEARCH_VOLUME_DIRECTORY:-.}/volumes-face-search/opensearch/data
目录赋权：chmod 777 ${FACESEARCH_VOLUME_DIRECTORY:-.}/volumes-face-search/opensearch/data
3、使用opensearch作为向量搜索引擎
docker-compose -f docker-compose-opensearch.yml --compatibility up -d
4、服务访问：
opensearch自带的可视化工具：http://127.0.0.1:5601
facesearch的swagger文档： http://127.0.0.1:56789/doc.html
```
* 项目编译
@ -101,17 +105,11 @@
* 项目中为了提高人脸的检出率，使用了主要和次要的人脸检测模型，目前实现了两种人脸检测模型insightface和PCN，在docker的服务中，默认主服务为PCN，备用服务为insightface。insightface的效率高，但针对于旋转了大角度的人脸检出率不高，而pcn则可以识别大角度旋转的图片，但效率低一些。若图像均为正脸的图像，建议使用insightface为主模型，pcn为备用模型，如何切换，请查看部署参数。
* 在测试过程中，针对milvus和proxima，发现proxima的速度比milvus稍快，但稳定性没有milvus好，线上服务使用时，还是建议使用milvus作为向量检索引擎。
### 项目演示
* 1.1.0 测试用例：face-search-test[测试用例-FaceSearchExample](https://gitee.com/open-visual/face-search/blob/master/face-search-test/src/main/java/com/visual/face/search/valid/exps/FaceSearchExample.java)
* 2.0.0 测试用例(做了优化，增强了搜索结果的区分度)：face-search-test[测试用例-FaceSearchExample](https://gitee.com/open-visual/face-search/blob/master/face-search-test/src/main/java/com/visual/face/search/valid/exps/FaceSearchExample.java)
* ![输入图片说明](scripts/images/validate.jpg)
* 1.2.0 测试用例(做了优化，增强了搜索结果的区分度)：face-search-test[测试用例-FaceSearchExample](https://gitee.com/open-visual/face-search/blob/master/face-search-test/src/main/java/com/visual/face/search/valid/exps/FaceSearchExample.java)
* ![输入图片说明](scripts/images/validate-1.2.0.jpg)
* ![输入图片说明](scripts/images/validate-2.0.0.jpg)
### 交流群

5
face-search-client/pom.xml Executable file → Normal file
View File

@ -2,12 +2,11 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>face-search-client</artifactId>
<groupId>com.visual.face.search</groupId>
<version>1.1.0</version>
<artifactId>face-search-client</artifactId>
<version>2.0.0</version>
<properties>
<java.version>1.8</java.version>

1
face-search-client/src/main/java/com/visual/face/search/handle/CollectHandler.java
View File

@ -46,7 +46,6 @@ public class CollectHandler extends BaseHandler<CollectHandler>{
.setMaxDocsPerSegment(collect.getMaxDocsPerSegment())
.setSampleColumns(collect.getSampleColumns())
.setFaceColumns(collect.getFaceColumns())
.setSyncBinLog(collect.isSyncBinLog())
.setShardsNum(collect.getShardsNum())
.setStorageFaceInfo(collect.getStorageFaceInfo())
.setStorageEngine(collect.getStorageEngine());

13
face-search-client/src/main/java/com/visual/face/search/model/Collect.java
View File

@ -18,8 +18,6 @@ public class Collect<ExtendsVo extends Collect<ExtendsVo>> implements Serializab
private List<FiledColumn> sampleColumns = new ArrayList<>();
/**自定义的人脸字段**/
private List<FiledColumn> faceColumns = new ArrayList<>();
/**启用binlog同步**/
private Boolean syncBinLog = false;
/**是否保留图片及人脸信息**/
private Boolean storageFaceInfo = false;
/**保留图片及人脸信息的存储组件**/
@ -86,17 +84,6 @@ public class Collect<ExtendsVo extends Collect<ExtendsVo>> implements Serializab
return (ExtendsVo) this;
}
public boolean isSyncBinLog() {
return null == syncBinLog ? false : syncBinLog;
}
public ExtendsVo setSyncBinLog(Boolean syncBinLog) {
if(null != syncBinLog){
this.syncBinLog = syncBinLog;
}
return (ExtendsVo) this;
}
public boolean getStorageFaceInfo() {
return null == storageFaceInfo ? false : storageFaceInfo;
}

10
face-search-client/src/main/java/com/visual/face/search/model/SampleFace.java
View File

@ -11,8 +11,6 @@ public class SampleFace implements Comparable<SampleFace>, Serializable {
/**人脸人数质量**/
private Float faceScore;
/**转换后的置信度**/
private Float distance;
/**转换后的置信度**/
private Float confidence;
/**样本扩展的额外数据**/
private KeyValues sampleData;
@ -67,14 +65,6 @@ public class SampleFace implements Comparable<SampleFace>, Serializable {
this.faceScore = faceScore;
}
public Float getDistance() {
return distance;
}
public void setDistance(Float distance) {
this.distance = distance;
}
public Float getConfidence() {
return confidence;
}

5
face-search-core/pom.xml Executable file → Normal file
View File

@ -5,8 +5,9 @@
<parent>
<artifactId>face-search</artifactId>
<groupId>com.visual.face.search</groupId>
<version>1.2.0</version>
<version>2.0.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>face-search-core</artifactId>
@ -30,6 +31,6 @@
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
</dependency>
</dependencies>
</project>

6
face-search-core/src/main/java/com/visual/face/search/core/domain/FaceInfo.java
View File

@ -123,9 +123,9 @@ public class FaceInfo implements Comparable<FaceInfo>, Serializable {
* @return 旋转后的角
*/
public Point rotation(Point center, float angle){
double k = new Float(Math.toRadians(angle));
float nx1 = new Float((this.x-center.x)*Math.cos(k) +(this.y-center.y)*Math.sin(k)+center.x);
float ny1 = new Float(-(this.x-center.x)*Math.sin(k) + (this.y-center.y)*Math.cos(k)+center.y);
double k = Math.toRadians(angle);
float nx1 = (float) ((this.x - center.x) * Math.cos(k) + (this.y - center.y) * Math.sin(k) + center.x);
float ny1 = (float) (-(this.x - center.x) * Math.sin(k) + (this.y - center.y) * Math.cos(k) + center.y);
return new Point(nx1, ny1);
}

2
face-search-core/src/main/java/com/visual/face/search/core/models/InsightArcFaceRecognition.java
View File

@ -42,7 +42,7 @@ public class InsightArcFaceRecognition extends BaseOnnxInfer implements FaceRec
.to4dFloatOnnxTensorAndDoReleaseMat(true);
output = getSession().run(Collections.singletonMap(getInputName(), tensor));
float[][] embeds = (float[][]) output.get(0).getValue();
return FaceInfo.Embedding.build(image.toBase64AndNoReleaseMat(), embeds[0]);
return Embedding.build(image.toBase64AndNoReleaseMat(), embeds[0]);
} catch (Exception e) {
throw new RuntimeException(e);
}finally {

10
face-search-core/src/main/java/com/visual/face/search/core/models/PcnNetworkFaceDetection.java
View File

@ -66,7 +66,7 @@ public class PcnNetworkFaceDetection extends BaseOnnxInfer implements FaceDetect
imgPad = pad_img_not_release_mat(mat);
float[] iouThs = iouTh <= 0 ? defIouThs : new float[]{iouTh, iouTh, 0.3f};
float[] scoreThs = scoreTh <= 0 ? defScoreThs : new float[]{0.375f * scoreTh, 0.5f * scoreTh, scoreTh};
List<PcnNetworkFaceDetection.Window2> willis = detect(this.getSessions(), mat, imgPad, scoreThs, iouThs);
List<Window2> willis = detect(this.getSessions(), mat, imgPad, scoreThs, iouThs);
return trans_window(mat, imgPad, willis);
} catch (Exception e) {
throw new RuntimeException(e);
@ -573,7 +573,7 @@ public class PcnNetworkFaceDetection extends BaseOnnxInfer implements FaceDetect
* @throws OrtException
* @throws IOException
*/
private static List<PcnNetworkFaceDetection.Window2> detect(OrtSession[] sessions, Mat img, Mat imgPad, float[] scoreThs, float iouThs[]) throws OrtException, IOException {
private static List<Window2> detect(OrtSession[] sessions, Mat img, Mat imgPad, float[] scoreThs, float iouThs[]) throws OrtException, IOException {
Mat img180 = new Mat();
Core.flip(imgPad, img180, 0);
@ -583,7 +583,7 @@ public class PcnNetworkFaceDetection extends BaseOnnxInfer implements FaceDetect
Mat imgNeg90 = new Mat();
Core.flip(img90, imgNeg90, 0);
List<PcnNetworkFaceDetection.Window2> winlist = stage1(img, imgPad, sessions[0], scoreThs[0]);
List<Window2> winlist = stage1(img, imgPad, sessions[0], scoreThs[0]);
winlist = NMS(winlist, true, iouThs[0]);
winlist = stage2(imgPad, img180, sessions[1], scoreThs[1], 24, winlist);
@ -604,7 +604,7 @@ public class PcnNetworkFaceDetection extends BaseOnnxInfer implements FaceDetect
/**
* 临时的人脸框
*/
private static class Window2 implements Comparable<PcnNetworkFaceDetection.Window2>{
private static class Window2 implements Comparable<Window2>{
public int x;
public int y;
public int w;
@ -624,7 +624,7 @@ public class PcnNetworkFaceDetection extends BaseOnnxInfer implements FaceDetect
}
@Override
public int compareTo(PcnNetworkFaceDetection.Window2 o) {
public int compareTo(Window2 o) {
if(o.conf == this.conf){
return new Integer(this.y).compareTo(o.y);
}else{

16
face-search-core/src/main/java/com/visual/face/search/core/utils/Similarity.java
View File

@ -76,4 +76,20 @@ public class Similarity {
return Double.valueOf(sim).floatValue();
}
/**
* 对cos的原始值进行进行增强
* @param cos
* @return
*/
public static float cosEnhance(float cos){
double sim = cos;
if(cos >= 0.5){
sim = cos + 2 * (cos - 0.5) * (1 - cos);
}else if(cos >= 0){
sim = cos - 2 * (cos - 0.5) * (0 - cos);
}
return Double.valueOf(sim).floatValue();
}
}

BIN
face-search-engine/libs/milvus-java-sdk-2.0.0.jar
View File

Binary file not shown.

BIN
face-search-engine/libs/milvus-java-sdk-2.0.4.jar
View File

Binary file not shown.

BIN
face-search-engine/libs/proxima-be-java-sdk-0.2.0.jar
View File

Binary file not shown.

104
face-search-engine/pom.xml Executable file → Normal file
View File

@ -5,118 +5,22 @@
<parent>
<artifactId>face-search</artifactId>
<groupId>com.visual.face.search</groupId>
<version>1.2.0</version>
<version>2.0.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>face-search-engine</artifactId>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<grpc.version>1.36.0</grpc.version>
<protobuf.version>3.12.0</protobuf.version>
<protoc.version>3.12.0</protoc.version>
<commons-collections4.version>4.3</commons-collections4.version>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-bom</artifactId>
<version>${grpc.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty-shaded</artifactId>
<version>1.36.0</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
<version>1.36.0</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
<version>1.36.0</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>21.0</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.16.16</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.14.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.30</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.12.1</version>
</dependency>
<dependency>
<groupId>io.milvus</groupId>
<artifactId>milvus-java-sdk</artifactId>
<version>2.0.4</version>
<scope>system</scope>
<systemPath>${project.basedir}/libs/milvus-java-sdk-2.0.4.jar</systemPath>
</dependency>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java-util</artifactId>
<version>${protobuf.version}</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-text</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>${commons-collections4.version}</version>
</dependency>
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20190722</version>
</dependency>
<dependency>
<groupId>com.alibaba.proxima</groupId>
<artifactId>proxima-be-java-sdk</artifactId>
<version>0.2.0</version>
<scope>system</scope>
<systemPath>${project.basedir}/libs/proxima-be-java-sdk-0.2.0.jar</systemPath>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-rest-high-level-client</artifactId>
</dependency>
</dependencies>
</project>

30
face-search-engine/src/main/java/com/visual/face/search/engine/api/SearchEngine.java Executable file
View File

@ -0,0 +1,30 @@
package com.visual.face.search.engine.api;
import com.visual.face.search.engine.model.MapParam;
import com.visual.face.search.engine.model.SearchResponse;
import java.util.List;
public interface SearchEngine {
public Object getEngine();
public boolean exist(String collectionName);
public boolean dropCollection(String collectionName);
public boolean createCollection(String collectionName, MapParam param);
public boolean insertVector(String collectionName, String sampleId, String faceId, float[] vectors);
public boolean deleteVectorByKey(String collectionName, String faceId);
public boolean deleteVectorByKey(String collectionName, List<String> faceIds);
public SearchResponse search(String collectionName, float[][] features, String algorithm, int topK);
public float searchMinScoreBySampleId(String collectionName, String sampleId,float[] feature, String algorithm);
public float searchMaxScoreBySampleId(String collectionName, String sampleId,float[] feature, String algorithm);
}

13
face-search-engine/src/main/java/com/visual/face/search/engine/conf/Constant.java Executable file
View File

@ -0,0 +1,13 @@
package com.visual.face.search.engine.conf;
public class Constant {
public final static String IndexShardsNum = "shardsNum";
public final static String IndexReplicasNum = "replicasNum";
public final static String ColumnNameFaceId = "face_id";
public final static String ColumnNameSampleId = "sample_id";
public final static String ColumnNameFaceVector = "face_vector";
public final static String ColumnNameFaceScore = "face_score";
}

25
face-search-engine/src/main/java/com/visual/face/search/engine/exps/SearchEngineException.java Normal file
View File

@ -0,0 +1,25 @@
package com.visual.face.search.engine.exps;
public class SearchEngineException extends RuntimeException{
public SearchEngineException() {
}
public SearchEngineException(String message) {
super(message);
}
public SearchEngineException(String message, Throwable cause) {
super(message, cause);
}
public SearchEngineException(Throwable cause) {
super(cause);
}
public SearchEngineException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
super(message, cause, enableSuppression, writableStackTrace);
}
}

247
face-search-engine/src/main/java/com/visual/face/search/engine/impl/OpenSearchEngine.java Normal file
View File

@ -0,0 +1,247 @@
package com.visual.face.search.engine.impl;
import com.visual.face.search.engine.api.SearchEngine;
import com.visual.face.search.engine.conf.Constant;
import com.visual.face.search.engine.exps.SearchEngineException;
import com.visual.face.search.engine.model.*;
import org.apache.commons.collections4.MapUtils;
import org.opensearch.action.DocWriteResponse;
import org.opensearch.action.admin.indices.delete.DeleteIndexRequest;
import org.opensearch.action.delete.DeleteRequest;
import org.opensearch.action.delete.DeleteResponse;
import org.opensearch.action.index.IndexRequest;
import org.opensearch.action.index.IndexResponse;
import org.opensearch.action.search.MultiSearchRequest;
import org.opensearch.action.search.MultiSearchResponse;
import org.opensearch.action.search.SearchRequest;
import org.opensearch.client.RequestOptions;
import org.opensearch.client.RestHighLevelClient;
import org.opensearch.client.indices.CreateIndexRequest;
import org.opensearch.client.indices.CreateIndexResponse;
import org.opensearch.client.indices.GetIndexRequest;
import org.opensearch.common.settings.Settings;
import org.opensearch.index.query.*;
import org.opensearch.index.query.functionscore.ScriptScoreQueryBuilder;
import org.opensearch.index.reindex.BulkByScrollResponse;
import org.opensearch.index.reindex.DeleteByQueryRequest;
import org.opensearch.rest.RestStatus;
import org.opensearch.script.Script;
import org.opensearch.search.SearchHit;
import org.opensearch.search.builder.SearchSourceBuilder;
import java.io.IOException;
import java.util.*;
public class OpenSearchEngine implements SearchEngine {
private RestHighLevelClient client;
private MapParam params = new MapParam();
public OpenSearchEngine(RestHighLevelClient client){
this(client, null);
}
public OpenSearchEngine(RestHighLevelClient client, MapParam params){
this.client = client;
if(null != params) { this.params = params; }
}
@Override
public Object getEngine() {
return this.client;
}
@Override
public boolean exist(String collectionName) {
try {
GetIndexRequest request = new GetIndexRequest(collectionName);
return this.client.indices().exists(request, RequestOptions.DEFAULT);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public boolean dropCollection(String collectionName) {
try {
DeleteIndexRequest request = new DeleteIndexRequest(collectionName);
return this.client.indices().delete(request, RequestOptions.DEFAULT).isAcknowledged();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public boolean createCollection(String collectionName, MapParam param) {
try {
//构建请求
CreateIndexRequest createIndexRequest = new CreateIndexRequest(collectionName);
createIndexRequest.settings(Settings.builder()
.put("index.number_of_shards", param.getIndexShardsNum())
.put("index.number_of_replicas", param.getIndexReplicasNum())
);
HashMap<String, Object> properties = new HashMap<>();
properties.put(Constant.ColumnNameSampleId, Map.of("type", "keyword"));
properties.put(Constant.ColumnNameFaceVector, Map.of("type", "knn_vector", "dimension", "512"));
createIndexRequest.mapping(Map.of("properties", properties));
//创建集合
CreateIndexResponse createIndexResponse = client.indices().create(createIndexRequest, RequestOptions.DEFAULT);
return createIndexResponse.isAcknowledged();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public boolean insertVector(String collectionName, String sampleId, String faceId, float[] vectors) {
try {
//构建请求
IndexRequest request = new IndexRequest(collectionName)
.id(faceId)
.source(Map.of(
Constant.ColumnNameSampleId, sampleId,
Constant.ColumnNameFaceVector, vectors
));
//插入数据
IndexResponse indexResponse = client.index(request, RequestOptions.DEFAULT);
DocWriteResponse.Result result = indexResponse.getResult();
return DocWriteResponse.Result.CREATED == result || DocWriteResponse.Result.UPDATED == result;
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public boolean deleteVectorByKey(String collectionName, String faceId) {
try {
DeleteRequest deleteDocumentRequest = new DeleteRequest(collectionName, faceId);
DeleteResponse deleteResponse = client.delete(deleteDocumentRequest, RequestOptions.DEFAULT);
return DocWriteResponse.Result.DELETED == deleteResponse.getResult();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public boolean deleteVectorByKey(String collectionName, List<String> keyIds) {
try {
String[] idArray = new String[keyIds.size()]; idArray = keyIds.toArray(idArray);
QueryBuilder queryBuilder = new BoolQueryBuilder().must(QueryBuilders.idsQuery().addIds(idArray));
DeleteByQueryRequest request = new DeleteByQueryRequest(collectionName).setQuery(queryBuilder);
BulkByScrollResponse response = client.deleteByQuery(request, RequestOptions.DEFAULT);
return response.getBulkFailures() != null && response.getBulkFailures().size() == 0;
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@Override
public SearchResponse search(String collectionName, float[][] features, String algorithm, int topK) {
try {
//构建搜索请求
MultiSearchRequest multiSearchRequest = new MultiSearchRequest();
for(float[] feature : features){
QueryBuilder queryBuilder = new MatchAllQueryBuilder();
Map<String, Object> params = new HashMap<>();
params.put("field", Constant.ColumnNameFaceVector);
params.put("space_type", algorithm);
params.put("query_value", feature);
Script script = new Script(Script.DEFAULT_SCRIPT_TYPE, "knn", "knn_score", params);
ScriptScoreQueryBuilder scriptScoreQueryBuilder = new ScriptScoreQueryBuilder(queryBuilder, script);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.query(scriptScoreQueryBuilder).size(topK)
.fetchSource(null, Constant.ColumnNameFaceVector); //是否需要向量字段
SearchRequest searchRequest = new SearchRequest(collectionName).source(searchSourceBuilder);
multiSearchRequest.add(searchRequest);
}
//查询索引
MultiSearchResponse response = this.client.msearch(multiSearchRequest, RequestOptions.DEFAULT);
MultiSearchResponse.Item[] responses = response.getResponses();
if(features.length != responses.length){
throw new SearchEngineException("features.length != responses.length");
}
//解析数据
List<SearchResult> result = new ArrayList<>();
for(MultiSearchResponse.Item item : response.getResponses()){
List<SearchDocument> documents = new ArrayList<>();
SearchHit[] searchHits = item.getResponse().getHits().getHits();
if(searchHits != null){
for(SearchHit searchHit : searchHits){
String faceId = searchHit.getId();
float score = searchHit.getScore()-1;
Map<String, Object> sourceMap = searchHit.getSourceAsMap();
String sampleId = MapUtils.getString(sourceMap, Constant.ColumnNameSampleId);
Object faceVector = MapUtils.getObject(sourceMap, Constant.ColumnNameFaceVector);
SearchDocument document = SearchDocument.build(sampleId, faceId, score).setVectors(faceVector);
documents.add(document);
}
}
result.add(SearchResult.build(documents));
}
//返回结果信息
return SearchResponse.build(SearchStatus.build(0, "success"), result);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override
public float searchMinScoreBySampleId(String collectionName, String sampleId,float[] feature, String algorithm) {
try {
//构建请求
QueryBuilder queryBuilder = new MatchQueryBuilder(Constant.ColumnNameSampleId, sampleId);
Map<String, Object> params = new HashMap<>();
params.put("field", Constant.ColumnNameFaceVector);
params.put("space_type", algorithm);
params.put("query_value", feature);
Script script = new Script(Script.DEFAULT_SCRIPT_TYPE, "knn", "knn_score", params);
ScriptScoreQueryBuilder scriptScoreQueryBuilder = new ScriptScoreQueryBuilder(queryBuilder, script);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.query(scriptScoreQueryBuilder)
.fetchSource(false).size(10000); //是否需要索引字段
SearchRequest searchRequest = new SearchRequest(collectionName).source(searchSourceBuilder);
//搜索请求
org.opensearch.action.search.SearchResponse response = this.client.search(searchRequest, RequestOptions.DEFAULT);
if(RestStatus.OK == response.status()){
SearchHit[] searchHits = response.getHits().getHits();
Double minScore = Arrays.stream(searchHits).mapToDouble(SearchHit::getScore).min().orElse(2f);
return minScore.floatValue()-1;
}else{
throw new RuntimeException("get score error!");
}
} catch (Exception e) {
throw new SearchEngineException(e);
}
}
@Override
public float searchMaxScoreBySampleId(String collectionName, String sampleId,float[] feature, String algorithm) {
try {
//构建请求
QueryBuilder queryBuilder = new BoolQueryBuilder()
.mustNot(new MatchQueryBuilder(Constant.ColumnNameSampleId, sampleId));
Map<String, Object> params = new HashMap<>();
params.put("field", Constant.ColumnNameFaceVector);
params.put("space_type", algorithm);
params.put("query_value", feature);
Script script = new Script(Script.DEFAULT_SCRIPT_TYPE, "knn", "knn_score", params);
ScriptScoreQueryBuilder scriptScoreQueryBuilder = new ScriptScoreQueryBuilder(queryBuilder, script);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.query(scriptScoreQueryBuilder)
.fetchSource(false).size(1); //是否需要索引字段
SearchRequest searchRequest = new SearchRequest(collectionName).source(searchSourceBuilder);
//搜索请求
org.opensearch.action.search.SearchResponse response = this.client.search(searchRequest, RequestOptions.DEFAULT);
if(RestStatus.OK == response.status()){
SearchHit[] searchHits = response.getHits().getHits();
Double maxScore = Arrays.stream(searchHits).mapToDouble(SearchHit::getScore).max().orElse(1f);
return maxScore.floatValue()-1;
}else{
throw new RuntimeException("get score error!");
}
} catch (Exception e) {
throw new SearchEngineException(e);
}
}
}

15
face-search-server/src/main/java/com/visual/face/search/server/engine/model/MapParam.java → face-search-engine/src/main/java/com/visual/face/search/engine/model/MapParam.java
View File

@ -1,6 +1,6 @@
package com.visual.face.search.server.engine.model;
package com.visual.face.search.engine.model;
import com.visual.face.search.server.engine.conf.Constant;
import com.visual.face.search.engine.conf.Constant;
import org.apache.commons.collections4.MapUtils;
import java.util.concurrent.ConcurrentHashMap;
@ -11,7 +11,6 @@ public class MapParam extends ConcurrentHashMap<String, Object> {
return new MapParam();
}
public MapParam put(String key, Object value){
if(null != key && null != value){
super.put(key, value);
@ -60,15 +59,15 @@ public class MapParam extends ConcurrentHashMap<String, Object> {
}
/******************************************************************************************************************/
public Long getMaxDocsPerSegment(){
Long maxDocsPerSegment = this.getLong(Constant.ParamKeyMaxDocsPerSegment, 0L);
public Long getIndexReplicasNum(){
Long maxDocsPerSegment = this.getLong(Constant.IndexReplicasNum, 1L);
maxDocsPerSegment = (null == maxDocsPerSegment || maxDocsPerSegment < 0) ? 0 : maxDocsPerSegment;
return maxDocsPerSegment;
}
public Integer getShardsNum(){
Integer shardsNum = this.getInteger(Constant.ParamKeyShardsNum, 0);
shardsNum = (null == shardsNum || shardsNum <= 0) ? 2 : shardsNum;
public Integer getIndexShardsNum(){
Integer shardsNum = this.getInteger(Constant.IndexShardsNum, 4);
shardsNum = (null == shardsNum || shardsNum <= 0) ? 4 : shardsNum;
return shardsNum;
}

72
face-search-engine/src/main/java/com/visual/face/search/engine/model/SearchDocument.java Executable file
View File

@ -0,0 +1,72 @@
package com.visual.face.search.engine.model;
import java.io.Serializable;
import com.visual.face.search.engine.utils.NumberUtils;
public class SearchDocument implements Serializable {
private float score;
private String faceId;
private String sampleId;
private Float[] vectors;
public SearchDocument(){}
public SearchDocument(String sampleId, String faceId, float score) {
this(sampleId, faceId, score, null);
}
public SearchDocument(String sampleId, String faceId, float score, Float[] vectors) {
this.score = score;
this.faceId = faceId;
this.sampleId = sampleId;
this.vectors = vectors;
}
public static SearchDocument build(String sampleId, String faceId, float score){
return build(sampleId, faceId, score, null);
}
public static SearchDocument build(String sampleId, String faceId, float score, Float[] vectors){
return new SearchDocument(sampleId, faceId, score, vectors);
}
public float getScore() {
return score;
}
public SearchDocument setScore(float score) {
this.score = score;
return this;
}
public String getFaceId() {
return faceId;
}
public SearchDocument setFaceId(String faceId) {
this.faceId = faceId;
return this;
}
public String getSampleId() {
return sampleId;
}
public SearchDocument setSampleId(String sampleId) {
this.sampleId = sampleId;
return this;
}
public Float[] getVectors() {
return vectors;
}
public SearchDocument setVectors(Object vectors) {
if(vectors == null){
return this;
}else{
this.vectors = NumberUtils.getFloatArray(vectors);
}
return this;
}
}

2
face-search-server/src/main/java/com/visual/face/search/server/engine/model/SearchResponse.java → face-search-engine/src/main/java/com/visual/face/search/engine/model/SearchResponse.java
View File

@ -1,4 +1,4 @@
package com.visual.face.search.server.engine.model;
package com.visual.face.search.engine.model;
import java.util.ArrayList;
import java.util.List;

2
face-search-server/src/main/java/com/visual/face/search/server/engine/model/SearchResult.java → face-search-engine/src/main/java/com/visual/face/search/engine/model/SearchResult.java
View File

@ -1,4 +1,4 @@
package com.visual.face.search.server.engine.model;
package com.visual.face.search.engine.model;
import java.util.ArrayList;

4
face-search-server/src/main/java/com/visual/face/search/server/engine/model/SearchStatus.java → face-search-engine/src/main/java/com/visual/face/search/engine/model/SearchStatus.java
View File

@ -1,8 +1,8 @@
package com.visual.face.search.server.engine.model;
package com.visual.face.search.engine.model;
public class SearchStatus {
private int code;
private String reason;
private String reason;
public SearchStatus(){}

74
face-search-engine/src/main/java/com/visual/face/search/engine/utils/NumberUtils.java Normal file
View File

@ -0,0 +1,74 @@
package com.visual.face.search.engine.utils;
import java.text.NumberFormat;
import java.text.ParseException;
import java.util.List;
public class NumberUtils {
public static Number getNumber(Object value) {
if (value != null) {
if (value instanceof Number) {
return (Number)value;
}
if (value instanceof String) {
try {
String text = (String)value;
return NumberFormat.getInstance().parse(text);
} catch (ParseException var4) {
}
}
}
return null;
}
public static Float getFloat(Object value) {
Number answer = getNumber(value);
if (answer == null) {
return null;
} else {
return answer instanceof Float ? (Float)answer : answer.floatValue();
}
}
public static Float[] getFloatArray(Object values) {
if(null != values){
if(values.getClass().isArray()){
return getFloatArray((Object[]) values);
}else if(values instanceof List){
return getFloatArray((List)values);
}else{
throw new RuntimeException("type error for:"+values.getClass());
}
}else{
return null;
}
}
public static Float[] getFloatArray(List<Object> values) {
if(null != values){
Float[] floats = new Float[values.size()];
for(int i=0; i<floats.length; i++){
floats[i] = getFloat(values.get(i));
}
return floats;
}else{
return null;
}
}
public static Float[] getFloatArray(Object[] values) {
if(null != values){
Float[] floats = new Float[values.length];
for(int i=0; i<floats.length; i++){
floats[i] = getFloat(values[i]);
}
return floats;
}else{
return null;
}
}
}

2
face-search-server/src/main/java/com/visual/face/search/server/engine/utils/VectorUtils.java → face-search-engine/src/main/java/com/visual/face/search/engine/utils/VectorUtils.java
View File

@ -1,4 +1,4 @@
package com.visual.face.search.server.engine.utils;
package com.visual.face.search.engine.utils;
import java.nio.ByteBuffer;
import java.util.ArrayList;

130
face-search-engine/src/test/java/com/visual/face/search/engine/test/SearchEngineTest.java Normal file
View File

@ -0,0 +1,130 @@
package com.visual.face.search.engine.test;
import com.visual.face.search.engine.impl.OpenSearchEngine;
import com.visual.face.search.engine.model.MapParam;
import com.visual.face.search.engine.model.SearchResponse;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.opensearch.client.RestClient;
import org.opensearch.client.RestClientBuilder;
import org.opensearch.client.RestHighLevelClient;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.X509Certificate;
import java.util.Arrays;
public class SearchEngineTest {
public static RestHighLevelClient getOpenSearchClientImpl(){
// final String hostName = "192.168.10.201";
final String hostName = "172.16.36.229";
final Integer hostPort = 9200;
final String hostScheme = "https";
final String userName = "admin";
final String password = "admin";
//认证参数
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(userName, password));
//ssl设置
final SSLContext sslContext;
try {
sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, new TrustManager[] { new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() { return null; }
public void checkClientTrusted(X509Certificate[] certs, String authType) {}
public void checkServerTrusted(X509Certificate[] certs, String authType) {}
}}, new SecureRandom());
} catch (NoSuchAlgorithmException | KeyManagementException e) {
throw new RuntimeException(e);
}
//构建请求
RestClientBuilder builder = RestClient.builder(new HttpHost(hostName, hostPort, hostScheme))
.setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setSSLHostnameVerifier((hostname, session) -> true)
.setSSLContext(sslContext)
.setMaxConnTotal(10)
.setMaxConnPerRoute(10)
);
//构建client
return new RestHighLevelClient(builder);
}
public static void main(String[] args) throws InterruptedException {
// String index_name = "app-opensearch-index-001";
// String index_name = "app-opensearch-index-002";
// String index_name = "python-test-index2";
String index_name = "visual_search_namespace_1_collect_20211201_v05_ohr6_vector";
OpenSearchEngine engine = new OpenSearchEngine(getOpenSearchClientImpl());
boolean exist = engine.exist(index_name);
System.out.println(exist);
if(exist){
// boolean drop = engine.dropCollection(index_name);
// System.out.println(drop);
}
// boolean create = engine.createCollection(index_name, MapParam.build());
// System.out.println(create);
//
// for(int i=0; i<10; i++){
// float[] vectors = new float[512];
// vectors[i] = 0.23333333f;
// boolean insert = engine.insertVector(index_name, "simple-0001", String.valueOf(i), vectors);
// System.out.println("insert="+insert);
// }
//
// Thread.sleep(2000);
// boolean delete = engine.deleteVectorByKey(index_name, "0");
// System.out.println(delete);
// boolean delete1 = engine.deleteVectorByKey(index_name, Arrays.asList("1", "5", "9"));
// System.out.println(delete1);
// float[][] a = new float[2][];
//
float[] vectors = new float[512];
vectors[0] = 0.768888f;
vectors[1] = 20000.768888f;
vectors[162] = 33333f;
// a[0] = vectors;
//
// float[] vectors1 = new float[512];
// vectors1[2] = 10000.54444f;
// a[1] = vectors1;
// SearchResponse searchResponse = engine.search(index_name, a, "cosinesimil",1);
// System.out.println(searchResponse);
// engine.searchCount(index_name, vectors, "", 1);
float minScore = engine.searchMinScoreBySampleId(index_name, "d4395b36984926a1934a0f9b916b32d21", vectors, "cosinesimil");
float maxScore = engine.searchMaxScoreBySampleId(index_name, "d4395b36984926a1934a0f9b916b32d21", vectors, "cosinesimil");
System.out.println(minScore);
System.out.println(maxScore);
System.exit(1);
// Object y = vectors;
// Object[] y1 = (Object[]) y;
// System.out.println(y1.length);
//
// System.out.println(y.getClass().getComponentType());
// System.out.println(y.getClass().isArray());
// System.out.println(cc instanceof Float);
// System.exit(1);
}
}

26
face-search-server/pom.xml Executable file → Normal file
View File

@ -5,7 +5,7 @@
<parent>
<artifactId>face-search</artifactId>
<groupId>com.visual.face.search</groupId>
<version>1.2.0</version>
<version>2.0.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
@ -17,6 +17,22 @@
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
<exclusions>
<exclusion>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
<dependency>
<groupId>com.visual.face.search</groupId>
<artifactId>face-search-core</artifactId>
@ -76,11 +92,11 @@
<!--文档插件-->
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<artifactId>springfox-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>com.github.xiaoymin</groupId>
<artifactId>swagger-bootstrap-ui</artifactId>
<artifactId>knife4j-spring-boot-starter</artifactId>
</dependency>
</dependencies>
@ -90,9 +106,7 @@
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<includeSystemScope>true</includeSystemScope>
</configuration>
<version>2.6.0</version>
</plugin>
</plugins>
</build>

85
face-search-server/src/main/java/com/visual/face/search/server/bootstrap/conf/EngineConfig.java
View File

@ -1,49 +1,74 @@
package com.visual.face.search.server.bootstrap.conf;
import com.alibaba.proxima.be.client.ConnectParam;
import com.alibaba.proxima.be.client.ProximaGrpcSearchClient;
import com.alibaba.proxima.be.client.ProximaSearchClient;
import com.visual.face.search.server.engine.api.SearchEngine;
import com.visual.face.search.server.engine.impl.MilvusSearchEngine;
import com.visual.face.search.server.engine.impl.ProximaSearchEngine;
import io.milvus.client.MilvusServiceClient;
import com.visual.face.search.engine.api.SearchEngine;
import com.visual.face.search.engine.impl.OpenSearchEngine;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.opensearch.client.RestClient;
import org.opensearch.client.RestClientBuilder;
import org.opensearch.client.RestHighLevelClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.security.cert.X509Certificate;
@Configuration("visualEngineConfig")
public class EngineConfig {
//日志
public Logger logger = LoggerFactory.getLogger(getClass());
@Value("${visual.engine.selected:proxima}")
private String selected;
@Value("${visual.engine.proxima.host}")
private String proximaHost;
@Value("${visual.engine.proxima.port:16000}")
private Integer proximaPort;
@Value("${visual.engine.milvus.host}")
private String milvusHost;
@Value("${visual.engine.milvus.port:19530}")
private Integer milvusPort;
@Value("${visual.engine.open-search.host:localhost}")
private String openSearchHost;
@Value("${visual.engine.open-search.port:9200}")
private Integer openSearchPort;
@Value("${visual.engine.open-search.scheme:https}")
private String openSearchScheme;
@Value("${visual.engine.open-search.username:admin}")
private String openSearchUserName;
@Value("${visual.engine.open-search.password:admin}")
private String openSearchPassword;
@Bean(name = "visualSearchEngine")
public SearchEngine getSearchEngine(){
if(selected.equalsIgnoreCase("milvus")){
logger.info("current vector engine is milvus");
io.milvus.param.ConnectParam connectParam = io.milvus.param.ConnectParam.newBuilder().withHost(milvusHost).withPort(milvusPort).build();
MilvusServiceClient client = new MilvusServiceClient(connectParam);
return new MilvusSearchEngine(client);
}else{
logger.info("current vector engine is proxima");
ConnectParam connectParam = ConnectParam.newBuilder().withHost(proximaHost).withPort(proximaPort).build();
ProximaSearchClient client = new ProximaGrpcSearchClient(connectParam);
return new ProximaSearchEngine(client);
public SearchEngine simpleSearchEngine(){
//认证参数
final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(openSearchUserName, openSearchPassword));
//ssl设置
final SSLContext sslContext;
try {
sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, new TrustManager[] { new X509TrustManager() {
public X509Certificate[] getAcceptedIssuers() { return null; }
public void checkClientTrusted(X509Certificate[] certs, String authType) {}
public void checkServerTrusted(X509Certificate[] certs, String authType) {}
}}, new SecureRandom());
} catch (NoSuchAlgorithmException | KeyManagementException e) {
logger.error("create SearchEngine error:", e);
throw new RuntimeException(e);
}
//构建请求
RestClientBuilder builder = RestClient.builder(new HttpHost(openSearchHost, openSearchPort, openSearchScheme))
.setHttpClientConfigCallback(httpClientBuilder -> httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setSSLHostnameVerifier((hostname, session) -> true)
.setSSLContext(sslContext)
.setMaxConnTotal(10)
.setMaxConnPerRoute(10)
);
//构建client
return new OpenSearchEngine(new RestHighLevelClient(builder));
}
}

39
face-search-server/src/main/java/com/visual/face/search/server/bootstrap/conf/Knife4jConfig.java Normal file
View File

@ -0,0 +1,39 @@
package com.visual.face.search.server.bootstrap.conf;
import org.springframework.context.annotation.Bean;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.builders.PathSelectors;
import springfox.documentation.builders.ApiInfoBuilder;
import springfox.documentation.spring.web.plugins.Docket;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import springfox.documentation.oas.annotations.EnableOpenApi;
import springfox.documentation.builders.RequestHandlerSelectors;
import com.github.xiaoymin.knife4j.spring.annotations.EnableKnife4j;
@Configuration
@EnableOpenApi
@EnableKnife4j
public class Knife4jConfig {
@Value("${visual.swagger.enable:true}")
private Boolean enable;
@Bean
public Docket createRestApi() {
return new Docket(DocumentationType.OAS_30)
.enable(enable)
.apiInfo(new ApiInfoBuilder()
.title("人脸搜索服务API")
.description("人脸搜索服务API")
.version("2.0.0")
.build())
.groupName("2.0.0")
.select()
.apis(RequestHandlerSelectors.basePackage("com.visual.face.search.server.controller.server"))
.paths(PathSelectors.any())
.build();
}
}

45
face-search-server/src/main/java/com/visual/face/search/server/bootstrap/conf/SwaggerConfig.java
View File

@ -1,45 +0,0 @@
package com.visual.face.search.server.bootstrap.conf;
import com.github.xiaoymin.swaggerbootstrapui.annotations.EnableSwaggerBootstrapUI;
import io.swagger.annotations.Api;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import springfox.documentation.builders.ApiInfoBuilder;
import springfox.documentation.builders.RequestHandlerSelectors;
import springfox.documentation.service.ApiInfo;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;
@Configuration
@EnableSwagger2
@EnableSwaggerBootstrapUI
public class SwaggerConfig implements WebMvcConfigurer {
@Value("${visual.swagger.enable:true}")
private Boolean enable;
@Bean
public Docket ProductApi() {
return new Docket(DocumentationType.SWAGGER_2)
.enable(enable)
.useDefaultResponseMessages(false)
.forCodeGeneration(false)
.pathMapping("/")
.apiInfo(apiInfo())
.select()
.apis(RequestHandlerSelectors.withClassAnnotation(Api.class))
.build();
}
private ApiInfo apiInfo() {
return new ApiInfoBuilder()
.title("人脸搜索服务API")
.description("人脸搜索服务API")
.version("1.1.0")
.build();
}
}

2
face-search-server/src/main/java/com/visual/face/search/server/config/DruidConfig.java
View File

@ -89,7 +89,7 @@ public class DruidConfig
Filter filter = new Filter()
{
@Override
public void init(javax.servlet.FilterConfig filterConfig) throws ServletException
public void init(FilterConfig filterConfig) throws ServletException
{
}

22
face-search-server/src/main/java/com/visual/face/search/server/domain/base/CollectVo.java
View File

@ -28,7 +28,7 @@ public class CollectVo<ExtendsVo extends CollectVo<ExtendsVo>> extends BaseVo {
/**数据分片中最大的文件个数**/
@Min(value = 0, message = "maxDocsPerSegment must greater than or equal to 0")
@ApiModelProperty(value="数据分片中最大的文件个数，默认为0（不限制）,仅对Proxima引擎生效", position = 3,required = false)
private Long maxDocsPerSegment;
private Long replicasNum;
/**数据分片中最大的文件个数**/
@Min(value = 0, message = "shardsNum must greater than or equal to 0")
@ApiModelProperty(value="要创建的集合的分片数，默认为0（即系统默认）,仅对Milvus引擎生效", position = 4,required = false)
@ -39,9 +39,6 @@ public class CollectVo<ExtendsVo extends CollectVo<ExtendsVo>> extends BaseVo {
/**自定义的人脸字段**/
@ApiModelProperty(value="自定义的人脸属性字段", position = 6,required = false)
private List<FiledColumn> faceColumns = new ArrayList<>();
/**启用binlog同步**/
@ApiModelProperty(value="启用binlog同步。扩展字段，暂不支持该功能", position = 7,required = false)
private Boolean syncBinLog;
/**是否保留图片及人脸信息**/
@ApiModelProperty(value="是否保留图片及人脸信息", position = 8,required = false)
private Boolean storageFaceInfo;
@ -86,12 +83,12 @@ public class CollectVo<ExtendsVo extends CollectVo<ExtendsVo>> extends BaseVo {
return (ExtendsVo) this;
}
public Long getMaxDocsPerSegment() {
return maxDocsPerSegment;
public Long getReplicasNum() {
return replicasNum;
}
public ExtendsVo setMaxDocsPerSegment(Long maxDocsPerSegment) {
this.maxDocsPerSegment = maxDocsPerSegment;
public ExtendsVo setReplicasNum(Long replicasNum) {
this.replicasNum = replicasNum;
return (ExtendsVo) this;
}
@ -126,15 +123,6 @@ public class CollectVo<ExtendsVo extends CollectVo<ExtendsVo>> extends BaseVo {
return (ExtendsVo) this;
}
public boolean isSyncBinLog() {
return null == syncBinLog ? false : syncBinLog;
}
public ExtendsVo setSyncBinLog(Boolean syncBinLog) {
this.syncBinLog = syncBinLog;
return (ExtendsVo) this;
}
public boolean getStorageFaceInfo() {
return null == storageFaceInfo ? false : storageFaceInfo;
}

17
face-search-server/src/main/java/com/visual/face/search/server/domain/extend/SampleFaceVo.java
View File

@ -16,16 +16,13 @@ public class SampleFaceVo implements Comparable<SampleFaceVo>, Serializable {
@ApiModelProperty(value="人脸分数:[0,100]", position = 3, required = true)
private Float faceScore;
/**转换后的置信度**/
@ApiModelProperty(value="向量距离:>=0", position = 4, required = true)
private Float distance;
/**转换后的置信度**/
@ApiModelProperty(value="转换后的置信度:[-100,100]，值越大，相似度越高。", position = 5, required = true)
@ApiModelProperty(value="转换后的置信度:[-100,100]，值越大，相似度越高。", position = 4, required = true)
private Float confidence;
/**样本扩展的额外数据**/
@ApiModelProperty(value="样本扩展的额外数据", position = 6, required = false)
@ApiModelProperty(value="样本扩展的额外数据", position = 5, required = false)
private FieldKeyValues sampleData;
/**人脸扩展的额外数据**/
@ApiModelProperty(value="人脸扩展的额外数据", position = 7, required = false)
@ApiModelProperty(value="人脸扩展的额外数据", position = 6, required = false)
private FieldKeyValues faceData;
/**
@ -76,14 +73,6 @@ public class SampleFaceVo implements Comparable<SampleFaceVo>, Serializable {
this.faceScore = faceScore;
}
public Float getDistance() {
return distance;
}
public void setDistance(Float distance) {
this.distance = distance;
}
public Float getConfidence() {
return confidence;
}

22
face-search-server/src/main/java/com/visual/face/search/server/domain/request/FaceSearchReqVo.java
View File

@ -30,13 +30,16 @@ public class FaceSearchReqVo extends BaseVo {
@Range(min = -100, max = 100, message = "faceScoreThreshold is not in the range")
@ApiModelProperty(value="人脸匹配分数阈值，范围：[-100,100]：默认0", position = 4, required = false)
private Float confidenceThreshold = 0f;
/**选择搜索评分的算法，默认余弦相似度(COSINESIMIL)，可选参数：L1、L2、LINF、COSINESIMIL、INNERPRODUCT、HAMMINGBIT**/
@ApiModelProperty(hidden = true, value="选择搜索评分的算法，默认是余弦相似度(COSINESIMIL)，可选参数：L1、L2、LINF、COSINESIMIL、INNERPRODUCT、HAMMINGBIT", position = 5, required = false)
private String algorithm = SearchAlgorithm.COSINESIMIL.name();
/**搜索条数：默认10**/
@Min(value = 0, message = "limit must greater than or equal to 0")
@ApiModelProperty(value="最大搜索条数：默认5", position = 5, required = false)
@ApiModelProperty(value="最大搜索条数：默认5", position = 6, required = false)
private Integer limit;
/**对输入图像中多少个人脸进行检索比对**/
@Min(value = 0, message = "maxFaceNum must greater than or equal to 0")
@ApiModelProperty(value="对输入图像中多少个人脸进行检索比对：默认5", position = 6, required = false)
@ApiModelProperty(value="对输入图像中多少个人脸进行检索比对：默认5", position = 7, required = false)
private Integer maxFaceNum;
/**
@ -95,6 +98,21 @@ public class FaceSearchReqVo extends BaseVo {
return this;
}
public SearchAlgorithm getAlgorithm() {
if(null != algorithm && !algorithm.isEmpty()){
return SearchAlgorithm.valueOf(this.algorithm);
}else{
return SearchAlgorithm.COSINESIMIL;
}
}
public FaceSearchReqVo setAlgorithm(String algorithm) {
if(null != algorithm){
this.algorithm = algorithm;
}
return this;
}
public Integer getLimit() {
return limit;
}

22
face-search-server/src/main/java/com/visual/face/search/server/domain/request/SearchAlgorithm.java Normal file
View File

@ -0,0 +1,22 @@
package com.visual.face.search.server.domain.request;
public enum SearchAlgorithm {
L1("l1"),
L2("l2"),
LINF("linf"),
HAMMINGBIT("innerproduct"),
INNERPRODUCT("hammingbit"),
COSINESIMIL("cosinesimil");
private String algorithm;
SearchAlgorithm(String algorithm){
this.algorithm = algorithm;
}
public String algorithm(){
return this.algorithm;
}
}

26
face-search-server/src/main/java/com/visual/face/search/server/engine/api/SearchEngine.java
View File

@ -1,26 +0,0 @@
package com.visual.face.search.server.engine.api;
import com.visual.face.search.server.engine.model.MapParam;
import com.visual.face.search.server.engine.model.SearchResponse;
import java.util.List;
public interface SearchEngine {
public Object getEngine();
public boolean exist(String collectionName);
public boolean dropCollection(String collectionName);
public boolean createCollection(String collectionName, MapParam param);
public boolean insertVector(String collectionName, Long keyId, String faceID, float[] vectors);
public boolean deleteVectorByKey(String collectionName, Long keyId);
public boolean deleteVectorByKey(String collectionName, List<Long> keyIds);
public SearchResponse search(String collectionName, float[][] features, int topK);
}

15
face-search-server/src/main/java/com/visual/face/search/server/engine/conf/Constant.java
View File

@ -1,15 +0,0 @@
package com.visual.face.search.server.engine.conf;
public class Constant {
public final static String ParamKeyShardsNum = "shardsNum";
public final static String ParamKeyMaxDocsPerSegment = "maxDocsPerSegment";
public final static String ColumnPrimaryKey = "id";
public final static String ColumnNameFaceId = "face_id";
public final static String ColumnNameFaceScore = "face_score";
public final static String ColumnNameFaceIndex = "face_index";
public final static String ColumnNameFaceVector = "face_vector";
public final static String ColumnNameSampleId = "sample_id";
}

199
face-search-server/src/main/java/com/visual/face/search/server/engine/impl/MilvusSearchEngine.java
View File

@ -1,199 +0,0 @@
package com.visual.face.search.server.engine.impl;
import com.visual.face.search.server.engine.api.SearchEngine;
import com.visual.face.search.server.engine.conf.Constant;
import com.visual.face.search.server.engine.model.*;
import com.visual.face.search.server.engine.utils.VectorUtils;
import io.milvus.response.SearchResultsWrapper;
import io.milvus.client.MilvusServiceClient;
import io.milvus.grpc.*;
import io.milvus.param.*;
import io.milvus.param.collection.*;
import io.milvus.param.dml.DeleteParam;
import io.milvus.param.dml.InsertParam;
import io.milvus.param.dml.SearchParam;
import io.milvus.param.index.CreateIndexParam;
import java.util.*;
public class MilvusSearchEngine implements SearchEngine {
private static final Integer SUCCESS_STATUE = 0;
private MilvusServiceClient client;
public MilvusSearchEngine(MilvusServiceClient client) {
this.client = client;
}
@Override
public Object getEngine(){
return this.client;
}
@Override
public boolean exist(String collectionName) {
HasCollectionParam param = HasCollectionParam.newBuilder().withCollectionName(collectionName).build();
R<Boolean> response = this.client.hasCollection(param);
if(SUCCESS_STATUE.equals(response.getStatus())){
return response.getData();
}else{
throw new RuntimeException(response.getMessage(), response.getException());
}
}
@Override
public boolean dropCollection(String collectionName) {
DropCollectionParam param = DropCollectionParam.newBuilder().withCollectionName(collectionName).build();
R<RpcStatus> response = this.client.dropCollection(param);
if(SUCCESS_STATUE.equals(response.getStatus())){
return true;
}else{
throw new RuntimeException(response.getMessage(), response.getException());
}
}
@Override
public boolean createCollection(String collectionName, MapParam param) {
FieldType keyFieldType = FieldType.newBuilder()
.withName(Constant.ColumnPrimaryKey)
.withDescription("id")
.withDataType(DataType.Int64)
.withPrimaryKey(true)
.withAutoID(false)
.build();
FieldType indexFieldType = FieldType.newBuilder()
.withName(Constant.ColumnNameFaceIndex)
.withDescription("face vector")
.withDataType(DataType.FloatVector)
.withDimension(512)
.build();
CreateCollectionParam createCollectionReq = CreateCollectionParam.newBuilder()
.withCollectionName(collectionName)
.withDescription(collectionName)
.addFieldType(keyFieldType)
.addFieldType(indexFieldType)
.withShardsNum(param.getShardsNum())
.build();
CreateIndexParam indexParam = CreateIndexParam.newBuilder()
.withCollectionName(collectionName)
.withFieldName(Constant.ColumnNameFaceIndex)
.withIndexType(IndexType.IVF_FLAT)
.withMetricType(MetricType.L2)
.withExtraParam("{\"nlist\":128}")
.withSyncMode(Boolean.TRUE)
.build();
LoadCollectionParam loadParam = LoadCollectionParam.newBuilder()
.withCollectionName(collectionName)
.build();
R<RpcStatus> response = this.client.createCollection(createCollectionReq);
if(SUCCESS_STATUE.equals(response.getStatus())){
R<RpcStatus> indexResponse = this.client.createIndex(indexParam);
if(SUCCESS_STATUE.equals(indexResponse.getStatus())){
R<RpcStatus> loadResponse = this.client.loadCollection(loadParam);
if(SUCCESS_STATUE.equals(loadResponse.getStatus())){
return true;
}else{
this.dropCollection(collectionName);
return false;
}
}else{
this.dropCollection(collectionName);
return false;
}
}else{
throw new RuntimeException(response.getMessage(), response.getException());
}
}
@Override
public boolean insertVector(String collectionName, Long keyId, String faceID, float[] vectors) {
List<InsertParam.Field> fields = new ArrayList<>();
fields.add(new InsertParam.Field(Constant.ColumnPrimaryKey, DataType.Int64, Collections.singletonList(keyId)));
fields.add(new InsertParam.Field(Constant.ColumnNameFaceIndex, DataType.FloatVector, Collections.singletonList(VectorUtils.convertVector(vectors))));
InsertParam insertParam = InsertParam.newBuilder()
.withCollectionName(collectionName)
.withFields(fields)
.build();
R<MutationResult> response = this.client.insert(insertParam);
if(SUCCESS_STATUE.equals(response.getStatus())){
return true;
}else{
throw new RuntimeException(response.getMessage(), response.getException());
}
}
@Override
public boolean deleteVectorByKey(String collectionName, Long keyId) {
String deleteExpr = Constant.ColumnPrimaryKey + " in " + "[" + keyId + "]";
DeleteParam build = DeleteParam.newBuilder()
.withCollectionName(collectionName)
.withExpr(deleteExpr)
.build();
R<MutationResult> response = this.client.delete(build);
if(SUCCESS_STATUE.equals(response.getStatus())){
return true;
}else{
throw new RuntimeException(response.getMessage(), response.getException());
}
}
@Override
public boolean deleteVectorByKey(String collectionName, List<Long> keyIds) {
String deleteExpr = Constant.ColumnPrimaryKey + " in " + keyIds.toString();
DeleteParam build = DeleteParam.newBuilder()
.withCollectionName(collectionName)
.withExpr(deleteExpr)
.build();
R<MutationResult> response = this.client.delete(build);
if(SUCCESS_STATUE.equals(response.getStatus())){
return true;
}else{
throw new RuntimeException(response.getMessage(), response.getException());
}
}
@Override
public SearchResponse search(String collectionName, float[][] features, int topK) {
SearchParam searchParam = SearchParam.newBuilder()
.withCollectionName(collectionName)
.withMetricType(MetricType.L2)
.withParams("{\"nprobe\": 128}")
.withOutFields(Collections.singletonList(Constant.ColumnPrimaryKey))
.withTopK(topK)
.withVectors(VectorUtils.convertVector(features))
.withVectorFieldName(Constant.ColumnNameFaceIndex)
.build();
R<SearchResults> response = this.client.search(searchParam);
if(SUCCESS_STATUE.equals(response.getStatus())){
SearchStatus status = SearchStatus.build(0, "success");
List<SearchResult> result = new ArrayList<>();
if(response.getData().getResults().hasIds()){
SearchResultsWrapper wrapper = new SearchResultsWrapper(response.getData().getResults());
for (int i = 0; i < features.length; ++i) {
List<SearchDocument> documents = new ArrayList<>();
List<SearchResultsWrapper.IDScore> scores = wrapper.getIDScore(i);
for(SearchResultsWrapper.IDScore scoreId : scores){
long primaryKey = scoreId.getLongID();
float score = scoreId.getScore();
documents.add(SearchDocument.build(primaryKey, score, null));
}
result.add(SearchResult.build(documents));
}
}
return SearchResponse.build(status, result);
}else{
throw new RuntimeException(response.getMessage(), response.getException());
}
}
}

174
face-search-server/src/main/java/com/visual/face/search/server/engine/impl/ProximaSearchEngine.java
View File

@ -1,174 +0,0 @@
package com.visual.face.search.server.engine.impl;
import com.alibaba.proxima.be.client.*;
import com.visual.face.search.server.engine.api.SearchEngine;
import com.visual.face.search.server.engine.conf.Constant;
import com.visual.face.search.server.engine.model.*;
import java.util.*;
public class ProximaSearchEngine implements SearchEngine {
private ProximaSearchClient client;
/**
* 构造获取连接对象
* @param client
*/
public ProximaSearchEngine(ProximaSearchClient client){
this.client = client;
}
@Override
public Object getEngine(){
return this.client;
}
@Override
public boolean exist(String collectionName) {
DescribeCollectionResponse response = client.describeCollection(collectionName);
return response.ok();
}
@Override
public boolean dropCollection(String collectionName) {
if(exist(collectionName)){
Status status = client.dropCollection(collectionName);
return status.ok();
}
return true;
}
@Override
public boolean createCollection(String collectionName, MapParam param) {
Long maxDocsPerSegment = param.getMaxDocsPerSegment();
CollectionConfig config = CollectionConfig.newBuilder()
.withCollectionName(collectionName)
.withMaxDocsPerSegment(maxDocsPerSegment)
.withForwardColumnNames(Arrays.asList(Constant.ColumnNameFaceId))
.addIndexColumnParam(Constant.ColumnNameFaceIndex, DataType.VECTOR_FP32, 512).build();
Status status = client.createCollection(config);
if(status.ok()){
return true;
}else{
throw new RuntimeException(status.getReason());
}
}
@Override
public boolean insertVector(String collectionName, Long keyId, String faceID, float[] vectors) {
WriteRequest.Row insertRow = WriteRequest.Row.newBuilder()
.withPrimaryKey(keyId)
.addIndexValue(vectors)
.addForwardValue(faceID)
.withOperationType(WriteRequest.OperationType.INSERT)
.build();
WriteRequest writeRequest = WriteRequest.newBuilder()
.withCollectionName(collectionName)
.withForwardColumnList(Collections.singletonList(Constant.ColumnNameFaceId))
.addIndexColumnMeta(Constant.ColumnNameFaceIndex, DataType.VECTOR_FP32, 512)
.addRow(insertRow)
.build();
Status status = client.write(writeRequest);
if(status.ok()){
return true;
}else{
throw new RuntimeException(status.getReason());
}
}
@Override
public boolean deleteVectorByKey(String collectionName, Long keyId) {
WriteRequest.Row deleteRow = WriteRequest.Row.newBuilder()
.withPrimaryKey(keyId)
.withOperationType(WriteRequest.OperationType.DELETE)
.build();
WriteRequest writeRequest = WriteRequest.newBuilder()
.withCollectionName(collectionName)
.withForwardColumnList(Collections.singletonList(Constant.ColumnNameFaceId))
.addIndexColumnMeta(Constant.ColumnNameFaceIndex, DataType.VECTOR_FP32, 512)
.addRow(deleteRow)
.build();
Status status = client.write(writeRequest);
if(status.ok()){
return true;
}else{
throw new RuntimeException(status.getReason());
}
}
@Override
public boolean deleteVectorByKey(String collectionName, List<Long> keyIds) {
if(null == keyIds || keyIds.isEmpty()){
return false;
}
List<Long> deleteIds = new ArrayList<>();
for(Long keyId : keyIds){
GetDocumentRequest r = GetDocumentRequest.newBuilder().withCollectionName(collectionName).withPrimaryKey(keyId).build();
GetDocumentResponse p = client.getDocumentByKey(r);
if(p.ok() && p.getDocument().getPrimaryKey() == keyId){
deleteIds.add(keyId);
}
}
if(deleteIds.isEmpty()){
return true;
}
WriteRequest.Builder builder = WriteRequest.newBuilder().withCollectionName(collectionName);
for(Long keyId : deleteIds){
WriteRequest.Row deleteRow = WriteRequest.Row.newBuilder().withPrimaryKey(keyId).withOperationType(WriteRequest.OperationType.DELETE).build();
builder.addRow(deleteRow);
}
Status status = client.write(builder.build());
if(status.ok()){
return true;
}else{
throw new RuntimeException(status.getReason());
}
}
@Override
public SearchResponse search(String collectionName, float[][] features, int topK) {
QueryRequest queryRequest = QueryRequest.newBuilder()
.withCollectionName(collectionName)
.withKnnQueryParam(
QueryRequest.KnnQueryParam.newBuilder()
.withColumnName(Constant.ColumnNameFaceIndex)
.withTopk(topK)
.withFeatures(features)
.build())
.build();
//搜索向量
QueryResponse queryResponse;
try {
queryResponse = client.query(queryRequest);
} catch (Exception e) {
return SearchResponse.build(SearchStatus.build(1, e.getMessage()), null);
}
//搜索失败
if(!queryResponse.ok()){
return SearchResponse.build(SearchStatus.build(1, "response status is error"), null);
}
//转换对象
List<SearchResult> result = new ArrayList<>();
SearchStatus status = SearchStatus.build(0, "success");
for (int i = 0; i < queryResponse.getQueryResultCount(); ++i) {
List<SearchDocument> documents = new ArrayList<>();
QueryResult queryResult = queryResponse.getQueryResult(i);
for (int d = 0; d < queryResult.getDocumentCount(); ++d) {
Document document = queryResult.getDocument(d);
long primaryKey = document.getPrimaryKey();
float score = document.getScore();
Set<String> forwardKeys = document.getForwardKeySet();
String faceId = null;
if(forwardKeys.contains(Constant.ColumnNameFaceId)){
faceId = document.getForwardValue(Constant.ColumnNameFaceId).getStringValue();
}
documents.add(SearchDocument.build(primaryKey, score, faceId));
}
result.add(SearchResult.build(documents));
}
//返回
return SearchResponse.build(status, result);
}
}

43
face-search-server/src/main/java/com/visual/face/search/server/engine/model/SearchDocument.java
View File

@ -1,43 +0,0 @@
package com.visual.face.search.server.engine.model;
public class SearchDocument {
private long primaryKey;
private float score;
private String faceId;
public SearchDocument(){}
public SearchDocument(long primaryKey, float score, String faceId) {
this.primaryKey = primaryKey;
this.score = score;
this.faceId = faceId;
}
public static SearchDocument build(long primaryKey, float score, String faceId){
return new SearchDocument(primaryKey, score, faceId);
}
public long getPrimaryKey() {
return primaryKey;
}
public void setPrimaryKey(long primaryKey) {
this.primaryKey = primaryKey;
}
public float getScore() {
return score;
}
public void setScore(float score) {
this.score = score;
}
public String getFaceId() {
return faceId;
}
public void setFaceId(String faceId) {
this.faceId = faceId;
}
}

6
face-search-server/src/main/java/com/visual/face/search/server/mapper/CollectMapper.java
View File

@ -44,7 +44,7 @@ public interface CollectMapper {
"and namespace = #{namespace,jdbcType=VARCHAR} ",
"and collection = #{collection,jdbcType=VARCHAR}"
})
int deleteByName(@Param("namespace")String namespace, @Param("collection")String collection);
int deleteByName(@Param("namespace") String namespace, @Param("collection") String collection);
@Select({
"select",
@ -72,7 +72,7 @@ public interface CollectMapper {
@Result(column="update_time", property="updateTime", jdbcType=JdbcType.TIMESTAMP),
@Result(column="deleted", property="deleted", jdbcType=JdbcType.CHAR)
})
Collection selectByName(@Param("namespace")String namespace, @Param("collection")String collection);
Collection selectByName(@Param("namespace") String namespace, @Param("collection") String collection);
@Select({
"select",
@ -98,5 +98,5 @@ public interface CollectMapper {
@Result(column="update_time", property="updateTime", jdbcType=JdbcType.TIMESTAMP),
@Result(column="deleted", property="deleted", jdbcType=JdbcType.CHAR)
})
List<Collection> selectByNamespace(@Param("namespace")String namespace);
List<Collection> selectByNamespace(@Param("namespace") String namespace);
}

30
face-search-server/src/main/java/com/visual/face/search/server/mapper/FaceDataMapper.java
View File

@ -61,25 +61,25 @@ public interface FaceDataMapper {
int deleteById(@Param("table") String table, @Param("id") Long id);
@Delete({"delete from ${table} where sample_id = #{sampleId,jdbcType=VARCHAR}"})
int deleteBySampleId(@Param("table") String table, @Param("sampleId")String sampleId);
int deleteBySampleId(@Param("table") String table, @Param("sampleId") String sampleId);
@Delete({"delete from ${table} where sample_id = #{sampleId,jdbcType=VARCHAR} and face_id=#{faceId,jdbcType=VARCHAR}"})
int deleteByFaceId(@Param("table") String table, @Param("sampleId")String sampleId, @Param("faceId")String faceId);
int deleteByFaceId(@Param("table") String table, @Param("sampleId") String sampleId, @Param("faceId") String faceId);
@Select({"select count(1) from ${table} where sample_id = '${sampleId}'"})
long countBySampleId(@Param("table") String table, @Param("sampleId")String sampleId);
long countBySampleId(@Param("table") String table, @Param("sampleId") String sampleId);
@Select({"select count(1) from ${table} where sample_id = '${sampleId}' and face_id=#{faceId,jdbcType=VARCHAR}"})
long count(@Param("table") String table, @Param("sampleId")String sampleId, @Param("faceId")String faceId);
long count(@Param("table") String table, @Param("sampleId") String sampleId, @Param("faceId") String faceId);
@Select({"select id from ${table} where sample_id = '${sampleId}' and face_id=#{faceId,jdbcType=VARCHAR}"})
Long getIdByFaceId(@Param("table") String table, @Param("sampleId")String sampleId, @Param("faceId")String faceId);
Long getIdByFaceId(@Param("table") String table, @Param("sampleId") String sampleId, @Param("faceId") String faceId);
@Select({"select id from ${table} where sample_id=#{sampleId,jdbcType=VARCHAR}"})
List<Long> getIdBySampleId(@Param("table") String table, @Param("sampleId")String sampleId);
@Select({"select face_id from ${table} where sample_id=#{sampleId,jdbcType=VARCHAR}"})
List<String> getFaceIdBySampleId(@Param("table") String table, @Param("sampleId") String sampleId);
@Select({"select * from ${table} where sample_id = #{sampleId,jdbcType=VARCHAR}"})
List<Map<String, Object>> getBySampleId(@Param("table") String table, @Param("sampleId")String sampleId);
List<Map<String, Object>> getBySampleId(@Param("table") String table, @Param("sampleId") String sampleId);
@Select({
"<script>",
@ -89,20 +89,24 @@ public interface FaceDataMapper {
"</foreach>",
"</script>"
})
List<Map<String, Object>> getBySampleIds(@Param("table") String table, @Param("sampleIds")List<String> sampleIds);
List<Map<String, Object>> getBySampleIds(@Param("table") String table, @Param("sampleIds") List<String> sampleIds);
@Select({"select * from ${table} where sample_id = #{sampleId,jdbcType=VARCHAR} and face_id=#{faceId,jdbcType=VARCHAR}"})
Map<String, Object> getByFaceId(@Param("table") String table, @Param("sampleId")String sampleId, @Param("faceId")String faceId);
Map<String, Object> getByFaceId(@Param("table") String table, @Param("sampleId") String sampleId, @Param("faceId") String faceId);
@Select({
"<script>",
"select * from ${table} where face_id in ",
"select",
"<foreach item=\"item\" index=\"index\" collection=\"columns\" open=\"\" separator=\",\" close=\"\">",
"${item}",
"</foreach>",
"from ${table} where face_id in ",
"<foreach collection=\"faceIds\" item=\"item\" index=\"index\" open=\"(\" separator=\",\" close=\")\">",
"#{item,jdbcType=VARCHAR}",
"</foreach>",
"</script>"
})
List<Map<String, Object>> getByFaceIds(@Param("table") String table, @Param("faceIds")List<String> faceIds);
List<Map<String, Object>> getByFaceIds(@Param("table") String table, @Param("columns") List<String> columns, @Param("faceIds") List<String> faceIds);
@Select({
"<script>",
@ -112,6 +116,6 @@ public interface FaceDataMapper {
"</foreach>",
"</script>"
})
List<Map<String, Object>> getByPrimaryIds(@Param("table") String table, @Param("keyIds")List<Long> keyIds);
List<Map<String, Object>> getByPrimaryIds(@Param("table") String table, @Param("keyIds") List<Long> keyIds);
}

2
face-search-server/src/main/java/com/visual/face/search/server/mapper/ImageDataMapper.java
View File

@ -18,6 +18,6 @@ public interface ImageDataMapper {
"#{image.createTime,jdbcType=TIMESTAMP}, #{image.modifyTime,jdbcType=TIMESTAMP})"
})
@SelectKey(statement="SELECT LAST_INSERT_ID()", keyProperty="id", before=false, resultType=Long.class)
int insert(@Param("table")String table, @Param("image")ImageData record);
int insert(@Param("table") String table, @Param("image") ImageData record);
}

4
face-search-server/src/main/java/com/visual/face/search/server/mapper/OperateTableMapper.java
View File

@ -12,10 +12,10 @@ import com.visual.face.search.server.model.TableColumn;
public interface OperateTableMapper {
@Select({"SHOW TABLES LIKE '${table}'"})
String showTable(@Param("table")String table);
String showTable(@Param("table") String table);
@Update({ "DROP TABLE IF EXISTS ${table}"})
int dropTable(@Param("table")String table);
int dropTable(@Param("table") String table);
@Update({
"<script>",

12
face-search-server/src/main/java/com/visual/face/search/server/mapper/SampleDataMapper.java
View File

@ -50,7 +50,7 @@ public interface SampleDataMapper {
int create(@Param("table") String table, @Param("sample") SampleData sample, @Param("columnValues") List<ColumnValue> columnValues);
@Select({"select count(1) from ${table} where sample_id = '${sampleId}'"})
long count(@Param("table") String table, @Param("sampleId")String sampleId);
long count(@Param("table") String table, @Param("sampleId") String sampleId);
@Update({
"<script>",
@ -79,16 +79,16 @@ public interface SampleDataMapper {
"where sample_id = #{sampleId,jdbcType=BIGINT}",
"</script>"
})
int update(@Param("table") String table, @Param("sampleId")String sampleId, @Param("columnValues") List<ColumnValue> columnValues);
int update(@Param("table") String table, @Param("sampleId") String sampleId, @Param("columnValues") List<ColumnValue> columnValues);
@Delete({"delete from ${table} where sample_id = #{sampleId,jdbcType=VARCHAR}",})
int delete(@Param("table") String table, @Param("sampleId")String sampleId);
int delete(@Param("table") String table, @Param("sampleId") String sampleId);
@Select({"select * from ${table} where sample_id = #{sampleId,jdbcType=VARCHAR}"})
Map<String, Object> getBySampleId(@Param("table") String table, @Param("sampleId")String sampleId);
Map<String, Object> getBySampleId(@Param("table") String table, @Param("sampleId") String sampleId);
@Select({"select * from ${table} order by id ${order} limit ${offset}, ${limit}"})
List<Map<String, Object>> getBySampleList(@Param("table") String table, @Param("offset")Integer offset, @Param("limit")Integer limit, @Param("order")String order);
List<Map<String, Object>> getBySampleList(@Param("table") String table, @Param("offset") Integer offset, @Param("limit") Integer limit, @Param("order") String order);
@Select({
"<script>",
@ -98,5 +98,5 @@ public interface SampleDataMapper {
"</foreach>",
"</script>"
})
List<Map<String, Object>> getBySampleIds(@Param("table") String table, @Param("sampleIds")List<String> sampleIds);
List<Map<String, Object>> getBySampleIds(@Param("table") String table, @Param("sampleIds") List<String> sampleIds);
}

158
face-search-server/src/main/java/com/visual/face/search/server/scheduler/FlushScheduler.java
View File

@ -1,79 +1,79 @@
package com.visual.face.search.server.scheduler;
import io.milvus.param.R;
import io.milvus.grpc.FlushResponse;
import io.milvus.client.MilvusServiceClient;
import io.milvus.param.collection.FlushParam;
import io.milvus.param.collection.HasCollectionParam;
import com.visual.face.search.server.utils.VTableCache;
import com.alibaba.proxima.be.client.ProximaSearchClient;
import com.visual.face.search.server.engine.api.SearchEngine;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import javax.annotation.Resource;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import org.springframework.scheduling.annotation.Scheduled;
/**
* 用于刷新数据，Milvus不刷新数据可能导致数据丢失
*/
@Component
public class FlushScheduler {
private static final Integer SUCCESS_STATUE = 0;
public Logger logger = LoggerFactory.getLogger(getClass());
@Resource
private SearchEngine searchEngine;
@Value("${visual.scheduler.flush.enable:true}")
private boolean enable;
@Scheduled(fixedDelayString = "${visual.scheduler.flush.interval:300000}")
public void flush(){
if(enable){
Object client = searchEngine.getEngine();
if(client instanceof MilvusServiceClient){
this.flushMilvus((MilvusServiceClient) client);
}else if(client instanceof ProximaSearchClient){
this.flushProxima((ProximaSearchClient) client);
}
}
}
/**
* 刷新数据到数据库中
* @param client
*/
private void flushMilvus(MilvusServiceClient client){
VTableCache.getVectorTables().forEach((vectorTable) -> {
try {
HasCollectionParam requestParam = HasCollectionParam.newBuilder().withCollectionName(vectorTable).build();
boolean exist = client.hasCollection(requestParam).getData();
if(exist){
FlushParam flushParam = FlushParam.newBuilder().addCollectionName(vectorTable).build();
R<FlushResponse> response = client.flush(flushParam);
if(SUCCESS_STATUE.equals(response.getStatus())){
VTableCache.remove(vectorTable);
logger.info("flushMilvus success: table is {}", vectorTable);
}else{
throw new RuntimeException("FlushResponse Error");
}
}
}catch (Exception e){
logger.error("flushMilvus error:", e);
}
});
}
/**
* 刷新数据到数据库中
* @param client
*/
private void flushProxima(ProximaSearchClient client){
}
}
//package com.visual.face.search.server.scheduler;
//
//import io.milvus.param.R;
//import io.milvus.grpc.FlushResponse;
//import io.milvus.client.MilvusServiceClient;
//import io.milvus.param.collection.FlushParam;
//import io.milvus.param.collection.HasCollectionParam;
//import com.visual.face.search.server.utils.VTableCache;
//import com.alibaba.proxima.be.client.ProximaSearchClient;
//import com.visual.face.search.server.engine.api.SearchEngine;
//
//import org.slf4j.Logger;
//import org.slf4j.LoggerFactory;
//import javax.annotation.Resource;
//
//import org.springframework.beans.factory.annotation.Value;
//import org.springframework.stereotype.Component;
//import org.springframework.scheduling.annotation.Scheduled;
//
///**
// * 用于刷新数据，Milvus不刷新数据可能导致数据丢失
// */
//@Component
//public class FlushScheduler {
//
// private static final Integer SUCCESS_STATUE = 0;
// public Logger logger = LoggerFactory.getLogger(getClass());
//
// @Resource
// private SearchEngine searchEngine;
// @Value("${visual.scheduler.flush.enable:true}")
// private boolean enable;
//
// @Scheduled(fixedDelayString = "${visual.scheduler.flush.interval:300000}")
// public void flush(){
// if(enable){
// Object client = searchEngine.getEngine();
// if(client instanceof MilvusServiceClient){
// this.flushMilvus((MilvusServiceClient) client);
// }else if(client instanceof ProximaSearchClient){
// this.flushProxima((ProximaSearchClient) client);
// }
// }
// }
//
// /**
// * 刷新数据到数据库中
// * @param client
// */
// private void flushMilvus(MilvusServiceClient client){
// VTableCache.getVectorTables().forEach((vectorTable) -> {
// try {
// HasCollectionParam requestParam = HasCollectionParam.newBuilder().withCollectionName(vectorTable).build();
// boolean exist = client.hasCollection(requestParam).getData();
// if(exist){
// FlushParam flushParam = FlushParam.newBuilder().addCollectionName(vectorTable).build();
// R<FlushResponse> response = client.flush(flushParam);
// if(SUCCESS_STATUE.equals(response.getStatus())){
// VTableCache.remove(vectorTable);
// logger.info("flushMilvus success: table is {}", vectorTable);
// }else{
// throw new RuntimeException("FlushResponse Error");
// }
// }
// }catch (Exception e){
// logger.error("flushMilvus error:", e);
// }
// });
// }
//
// /**
// * 刷新数据到数据库中
// * @param client
// */
// private void flushProxima(ProximaSearchClient client){
//
// }
//
//}

13
face-search-server/src/main/java/com/visual/face/search/server/service/impl/CollectServiceImpl.java
View File

@ -5,10 +5,9 @@ import com.visual.face.search.core.utils.JsonUtil;
import com.visual.face.search.core.utils.ThreadUtil;
import com.visual.face.search.server.domain.request.CollectReqVo;
import com.visual.face.search.server.domain.response.CollectRepVo;
import com.visual.face.search.server.engine.api.SearchEngine;
import com.visual.face.search.server.engine.conf.Constant;
import com.visual.face.search.server.engine.model.MapParam;
//import com.visual.face.search.server.mapper.CollectMapper;
import com.visual.face.search.engine.api.SearchEngine;
import com.visual.face.search.engine.conf.Constant;
import com.visual.face.search.engine.model.MapParam;
import com.visual.face.search.server.mapper.CollectMapper;
import com.visual.face.search.server.model.Collection;
import com.visual.face.search.server.service.api.CollectService;
@ -21,12 +20,10 @@ import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.transaction.interceptor.TransactionAspectSupport;
import org.springframework.util.DigestUtils;
import javax.annotation.Resource;
import java.util.ArrayList;
import java.util.List;
import java.util.UUID;
@Service("visualCollectService")
@ -91,8 +88,8 @@ public class CollectServiceImpl extends BaseService implements CollectService {
}
//创建人脸向量库
MapParam param = MapParam.build()
.put(Constant.ParamKeyShardsNum, collect.getShardsNum())
.put(Constant.ParamKeyMaxDocsPerSegment, collect.getMaxDocsPerSegment());
.put(Constant.IndexShardsNum, collect.getShardsNum())
.put(Constant.IndexReplicasNum, collect.getReplicasNum());
boolean createVectorFlag = searchEngine.createCollection(vectorTableName, param);
if(!createVectorFlag){
throw new RuntimeException("create vector table error");

57
face-search-server/src/main/java/com/visual/face/search/server/service/impl/FaceDataServiceImpl.java
View File

@ -1,12 +1,13 @@
package com.visual.face.search.server.service.impl;
import com.visual.face.search.engine.api.SearchEngine;
import com.visual.face.search.core.domain.ExtParam;
import com.visual.face.search.core.domain.FaceImage;
import com.visual.face.search.core.domain.FaceInfo;
import com.visual.face.search.core.domain.ImageMat;
import com.visual.face.search.core.extract.FaceFeatureExtractor;
import com.visual.face.search.core.utils.JsonUtil;
import com.visual.face.search.core.utils.Similarity;
import com.visual.face.search.server.domain.request.SearchAlgorithm;
import com.visual.face.search.server.domain.storage.StorageDataInfo;
import com.visual.face.search.server.domain.storage.StorageImageInfo;
import com.visual.face.search.server.domain.storage.StorageInfo;
@ -14,11 +15,6 @@ import com.visual.face.search.server.domain.extend.FieldKeyValue;
import com.visual.face.search.server.domain.extend.FieldKeyValues;
import com.visual.face.search.server.domain.request.FaceDataReqVo;
import com.visual.face.search.server.domain.response.FaceDataRepVo;
import com.visual.face.search.server.engine.api.SearchEngine;
import com.visual.face.search.server.engine.conf.Constant;
import com.visual.face.search.server.engine.model.SearchDocument;
import com.visual.face.search.server.engine.model.SearchResponse;
import com.visual.face.search.server.engine.model.SearchResult;
import com.visual.face.search.server.mapper.CollectMapper;
import com.visual.face.search.server.mapper.FaceDataMapper;
import com.visual.face.search.server.mapper.SampleDataMapper;
@ -33,8 +29,6 @@ import com.visual.face.search.server.service.base.BaseService;
import com.visual.face.search.server.utils.CollectionUtil;
import com.visual.face.search.server.utils.TableUtils;
import com.visual.face.search.server.utils.VTableCache;
import com.visual.face.search.server.utils.ValueUtil;
import org.apache.commons.collections4.MapUtils;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
@ -101,45 +95,18 @@ public class FaceDataServiceImpl extends BaseService implements FaceDataService
float[] embeds = faceInfo.embedding.embeds;
//当前样本的人脸相似度的最小阈值
if(null != face.getMinConfidenceThresholdWithThisSample() && face.getMinConfidenceThresholdWithThisSample() > 0){
List<Map<String, Object>> faces = faceDataMapper.getBySampleId(collection.getFaceTable(), face.getSampleId());
for(Map<String, Object> item : faces){
String faceVectorStr = MapUtils.getString(item, Constant.ColumnNameFaceVector);
float[] faceVector = ValueUtil.convertVector(faceVectorStr);
float simVal = Similarity.cosineSimilarityNorm(embeds, faceVector);
float confidence = (float) Math.floor(simVal * 10000)/100;
if(confidence < face.getMinConfidenceThresholdWithThisSample()){
throw new RuntimeException("this face confidence is less than minConfidenceThresholdWithThisSample,confidence="+confidence+",threshold="+face.getMinConfidenceThresholdWithThisSample());
}
float minScore = this.searchEngine.searchMinScoreBySampleId(collection.getVectorTable(), face.getSampleId(), embeds, SearchAlgorithm.COSINESIMIL.algorithm());
float confidence = (float) Math.floor(minScore * 10000)/100;
if(confidence < face.getMinConfidenceThresholdWithThisSample()){
throw new RuntimeException("this face confidence is less than minConfidenceThresholdWithThisSample,confidence="+confidence+",threshold="+face.getMinConfidenceThresholdWithThisSample());
}
}
//当前样本与其他样本的人脸相似度的最大阈值
if(null != face.getMaxConfidenceThresholdWithOtherSample() && face.getMaxConfidenceThresholdWithOtherSample() > 0){
//查询
List<Long> otherFaceIds = new ArrayList<>();
List<Long> faceIds = faceDataMapper.getIdBySampleId(collection.getFaceTable(), face.getSampleId());
int topK = faceIds.size() + 2;
float [][] vectors = new float[1][]; vectors[0] = embeds;
SearchResponse response = searchEngine.search(collection.getVectorTable(), vectors, topK);
if(response.getStatus().ok()){
for(SearchResult result : response.getResult()){
for(SearchDocument document : result.getDocuments()){
if(!faceIds.contains(document.getPrimaryKey())){
otherFaceIds.add(document.getPrimaryKey());
}
}
}
}
if(!otherFaceIds.isEmpty()){
List<Map<String, Object>> faces = faceDataMapper.getByPrimaryIds(collection.getFaceTable(), otherFaceIds);
for(Map<String, Object> item : faces){
String faceVectorStr = MapUtils.getString(item, Constant.ColumnNameFaceVector);
float[] faceVector = ValueUtil.convertVector(faceVectorStr);
float simVal = Similarity.cosineSimilarityNorm(embeds, faceVector);
float confidence = (float) Math.floor(simVal * 10000)/100;
if(confidence > face.getMaxConfidenceThresholdWithOtherSample()){
throw new RuntimeException("this face confidence is gather than maxConfidenceThresholdWithOtherSample,confidence="+confidence+",threshold="+face.getMaxConfidenceThresholdWithOtherSample());
}
}
float minScore = this.searchEngine.searchMaxScoreBySampleId(collection.getVectorTable(), face.getSampleId(), embeds, SearchAlgorithm.COSINESIMIL.algorithm());
float confidence = (float) Math.floor(minScore * 10000)/100;
if(confidence > face.getMaxConfidenceThresholdWithOtherSample()){
throw new RuntimeException("this face confidence is gather than maxConfidenceThresholdWithOtherSample,confidence="+confidence+",threshold="+face.getMaxConfidenceThresholdWithOtherSample());
}
}
//保存图片信息并获取图片存储
@ -188,7 +155,7 @@ public class FaceDataServiceImpl extends BaseService implements FaceDataService
throw new RuntimeException("create face error");
}
//写入数据到人脸向量库
boolean flag1 = searchEngine.insertVector(collection.getVectorTable(), facePo.getId(), faceId, embeds);
boolean flag1 = searchEngine.insertVector(collection.getVectorTable(), face.getSampleId(), faceId, embeds);
if(!flag1){
throw new RuntimeException("create face vector error");
}
@ -218,7 +185,7 @@ public class FaceDataServiceImpl extends BaseService implements FaceDataService
throw new RuntimeException("face id is not exist");
}
//删除向量
boolean delete = searchEngine.deleteVectorByKey(collection.getVectorTable(), keyId);
boolean delete = searchEngine.deleteVectorByKey(collection.getVectorTable(), faceId);
if(!delete){
throw new RuntimeException("delete face vector error");
}

44
face-search-server/src/main/java/com/visual/face/search/server/service/impl/FaceSearchServiceImpl.java
View File

@ -12,12 +12,13 @@ import com.visual.face.search.core.utils.Similarity;
import com.visual.face.search.server.domain.extend.FaceLocation;
import com.visual.face.search.server.domain.extend.SampleFaceVo;
import com.visual.face.search.server.domain.request.FaceSearchReqVo;
import com.visual.face.search.server.domain.request.SearchAlgorithm;
import com.visual.face.search.server.domain.response.FaceSearchRepVo;
import com.visual.face.search.server.engine.api.SearchEngine;
import com.visual.face.search.server.engine.conf.Constant;
import com.visual.face.search.server.engine.model.SearchDocument;
import com.visual.face.search.server.engine.model.SearchResponse;
import com.visual.face.search.server.engine.model.SearchResult;
import com.visual.face.search.engine.api.SearchEngine;
import com.visual.face.search.engine.conf.Constant;
import com.visual.face.search.engine.model.SearchDocument;
import com.visual.face.search.engine.model.SearchResponse;
import com.visual.face.search.engine.model.SearchResult;
import com.visual.face.search.server.mapper.CollectMapper;
import com.visual.face.search.server.mapper.FaceDataMapper;
import com.visual.face.search.server.mapper.SampleDataMapper;
@ -82,7 +83,7 @@ public class FaceSearchServiceImpl extends BaseService implements FaceSearchServ
}
//特征搜索
int topK = (null == search.getLimit() || search.getLimit() <= 0) ? 5 : search.getLimit();
SearchResponse searchResponse =searchEngine.search(collection.getVectorTable(), vectors, topK);
SearchResponse searchResponse =searchEngine.search(collection.getVectorTable(), vectors, search.getAlgorithm().algorithm(), topK);
if(!searchResponse.getStatus().ok()){
throw new RuntimeException(searchResponse.getStatus().getReason());
}
@ -106,33 +107,19 @@ public class FaceSearchServiceImpl extends BaseService implements FaceSearchServ
return vos;
}
//获取关联数据ID
boolean needFixFaceId = false;
Set<Long> faceIds = new HashSet<>();
Set<String> faceIds = new HashSet<>();
for(SearchResult searchResult : result){
List<SearchDocument> documents = searchResult.getDocuments();
for(SearchDocument document : documents){
faceIds.add(document.getPrimaryKey());
if(null == document.getFaceId() || document.getFaceId().isEmpty()){
needFixFaceId = true;
}
faceIds.add(document.getFaceId());
}
}
//查询数据
List<Map<String, Object>> faceList = faceDataMapper.getByPrimaryIds(collection.getFaceTable(), new ArrayList<>(faceIds));
List<Map<String, Object>> faceList = faceDataMapper.getByFaceIds(collection.getFaceTable(), ValueUtil.getAllFaceColumnNames(collection), new ArrayList<>(faceIds));
Set<String> sampleIds = faceList.stream().map(item -> MapUtils.getString(item, Constant.ColumnNameSampleId)).collect(Collectors.toSet());
List<Map<String, Object>> sampleList = sampleDataMapper.getBySampleIds(collection.getSampleTable(), new ArrayList<>(sampleIds));
Map<String, Map<String, Object>> faceMapping = ValueUtil.mapping(faceList, Constant.ColumnNameFaceId);
Map<String, Map<String, Object>> sampleMapping = ValueUtil.mapping(sampleList, Constant.ColumnNameSampleId);
//补全结果数据中的FaceId。由于milvus不支持字符串结构，只会返回人脸数据的主键ID
if(needFixFaceId){
Map<Long, String> mapping = ValueUtil.mapping(faceList, Constant.ColumnPrimaryKey, Constant.ColumnNameFaceId);
for(SearchResult searchResult : result){
List<SearchDocument> documents = searchResult.getDocuments();
for(SearchDocument document : documents){
document.setFaceId(mapping.get(document.getPrimaryKey()));
}
}
}
//构造返回结果
List<FaceSearchRepVo> vos = new ArrayList<>();
for(int i=0; i<faceInfos.size(); i++){
@ -148,10 +135,12 @@ public class FaceSearchServiceImpl extends BaseService implements FaceSearchServ
if(null != face){
float faceScore = MapUtils.getFloatValue(face, Constant.ColumnNameFaceScore);
String sampleId = MapUtils.getString(face, Constant.ColumnNameSampleId);
String faceVectorStr = MapUtils.getString(face, Constant.ColumnNameFaceVector);
float[] faceVector = ValueUtil.convertVector(faceVectorStr);
float simVal = Similarity.cosineSimilarityNorm(faceInfos.get(i).embedding.embeds, faceVector);
float confidence = (float) Math.floor(simVal * 1000000)/10000;
float score = document.getScore();
float confidence = score;
if(SearchAlgorithm.COSINESIMIL == search.getAlgorithm()){
score = Similarity.cosEnhance(score);
confidence = (float) Math.floor(score * 1000000)/10000;
}
if(null != sampleId && sampleMapping.containsKey(sampleId) && confidence >= search.getConfidenceThreshold()){
Map<String, Object> sample = sampleMapping.get(sampleId);
SampleFaceVo faceVo = SampleFaceVo.build();
@ -159,7 +148,6 @@ public class FaceSearchServiceImpl extends BaseService implements FaceSearchServ
faceVo.setFaceId(document.getFaceId());
faceVo.setFaceScore(faceScore);
faceVo.setConfidence(confidence);
faceVo.setDistance((float) Math.floor(document.getScore() * 10000) / 10000);
faceVo.setFaceData(ValueUtil.getFieldKeyValues(face, ValueUtil.getFaceColumns(collection)));
faceVo.setSampleData(ValueUtil.getFieldKeyValues(sample, ValueUtil.getSampleColumns(collection)));
match.add(faceVo);

7
face-search-server/src/main/java/com/visual/face/search/server/service/impl/SampleDataServiceImpl.java
View File

@ -1,12 +1,12 @@
package com.visual.face.search.server.service.impl;
import com.visual.face.search.engine.conf.Constant;
import com.visual.face.search.engine.api.SearchEngine;
import com.visual.face.search.server.domain.extend.FieldKeyValue;
import com.visual.face.search.server.domain.extend.FieldKeyValues;
import com.visual.face.search.server.domain.extend.SimpleFaceVo;
import com.visual.face.search.server.domain.request.SampleDataReqVo;
import com.visual.face.search.server.domain.response.SampleDataRepVo;
import com.visual.face.search.server.engine.api.SearchEngine;
import com.visual.face.search.server.engine.conf.Constant;
import com.visual.face.search.server.mapper.CollectMapper;
import com.visual.face.search.server.mapper.FaceDataMapper;
import com.visual.face.search.server.mapper.SampleDataMapper;
@ -21,7 +21,6 @@ import org.apache.commons.collections4.MapUtils;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import javax.annotation.Resource;
import java.util.*;
import java.util.stream.Collectors;
@ -128,7 +127,7 @@ public class SampleDataServiceImpl extends BaseService implements SampleDataServ
throw new RuntimeException("sample_id is not exist");
}
//删除向量数据
List<Long> faceIds = faceDataMapper.getIdBySampleId(collection.getFaceTable(), sampleId);
List<String> faceIds = faceDataMapper.getFaceIdBySampleId(collection.getFaceTable(), sampleId);
searchEngine.deleteVectorByKey(collection.getVectorTable(), faceIds);
//删除人脸数据
faceDataMapper.deleteBySampleId(collection.getFaceTable(), sampleId);

10
face-search-server/src/main/java/com/visual/face/search/server/utils/SpringUtils.java
View File

@ -39,7 +39,7 @@ public final class SpringUtils implements BeanFactoryPostProcessor, ApplicationC
*
* @param name
* @return Object 一个以所给名字注册的bean的实例
* @throws org.springframework.beans.BeansException
* @throws BeansException
*
*/
@SuppressWarnings("unchecked")
@ -53,7 +53,7 @@ public final class SpringUtils implements BeanFactoryPostProcessor, ApplicationC
*
* @param clz
* @return
* @throws org.springframework.beans.BeansException
* @throws BeansException
*
*/
public static <T> T getBean(Class<T> clz) throws BeansException
@ -78,7 +78,7 @@ public final class SpringUtils implements BeanFactoryPostProcessor, ApplicationC
*
* @param name
* @return boolean
* @throws org.springframework.beans.factory.NoSuchBeanDefinitionException
* @throws NoSuchBeanDefinitionException
*
*/
public static boolean isSingleton(String name) throws NoSuchBeanDefinitionException
@ -89,7 +89,7 @@ public final class SpringUtils implements BeanFactoryPostProcessor, ApplicationC
/**
* @param name
* @return Class 注册对象的类型
* @throws org.springframework.beans.factory.NoSuchBeanDefinitionException
* @throws NoSuchBeanDefinitionException
*
*/
public static Class<?> getType(String name) throws NoSuchBeanDefinitionException
@ -102,7 +102,7 @@ public final class SpringUtils implements BeanFactoryPostProcessor, ApplicationC
*
* @param name
* @return
* @throws org.springframework.beans.factory.NoSuchBeanDefinitionException
* @throws NoSuchBeanDefinitionException
*
*/
public static String[] getAliases(String name) throws NoSuchBeanDefinitionException

17
face-search-server/src/main/java/com/visual/face/search/server/utils/ValueUtil.java
View File

@ -1,9 +1,11 @@
package com.visual.face.search.server.utils;
import com.visual.face.search.core.utils.JsonUtil;
import com.visual.face.search.engine.conf.Constant;
import com.visual.face.search.server.domain.extend.FieldKeyValue;
import com.visual.face.search.server.domain.extend.FieldKeyValues;
import com.visual.face.search.server.domain.extend.FiledColumn;
import com.visual.face.search.server.domain.extend.FiledDataType;
import com.visual.face.search.server.domain.response.CollectRepVo;
import com.visual.face.search.server.model.Collection;
import org.apache.commons.collections4.MapUtils;
@ -12,6 +14,7 @@ import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
public class ValueUtil {
@ -25,6 +28,7 @@ public class ValueUtil {
return new ArrayList<>();
}
public static List<FiledColumn> getSampleColumns(Collection collection){
if(null != collection.getSchemaInfo() && !collection.getSchemaInfo().isEmpty()){
CollectRepVo collectVo = JsonUtil.toEntity(collection.getSchemaInfo(), CollectRepVo.class);
@ -35,6 +39,19 @@ public class ValueUtil {
return new ArrayList<>();
}
public static List<String> getAllFaceColumnNames(Collection collection){
List<FiledColumn> columns = getAllFaceColumns(collection);
return columns.stream().map(FiledColumn::getName).collect(Collectors.toList());
}
public static List<FiledColumn> getAllFaceColumns(Collection collection){
List<FiledColumn> columns = getFaceColumns(collection);
columns.add(FiledColumn.build().setName(Constant.ColumnNameSampleId).setComment("样本ID").setDataType(FiledDataType.STRING));
columns.add(FiledColumn.build().setName(Constant.ColumnNameFaceId).setComment("人脸ID").setDataType(FiledDataType.STRING));
columns.add(FiledColumn.build().setName(Constant.ColumnNameFaceScore).setComment("人脸分数").setDataType(FiledDataType.FLOAT));
return columns;
}
public static FieldKeyValues getFieldKeyValues(Map<String, Object> map , List<FiledColumn> columns){
columns = null != columns ? columns : new ArrayList<>();
Map<String, String> keyMap = new HashMap<>();

13
face-search-server/src/main/resources/application-dev.yml
View File

@ -47,13 +47,12 @@ visual:
modelPath:
thread: 1
engine:
selected: milvus
proxima:
host: visual-face-search-proxima
port: 16000
milvus:
host: visual-face-search-milvus
port: 19530
open-search:
host: visual-face-search-opensearch
port: 9200
scheme: https
username: admin
password: admin
scheduler:
flush:
enable: true

57
face-search-server/src/main/resources/application-docker.yml
View File

@ -23,37 +23,36 @@ logging:
visual:
model:
faceDetection:
name: InsightScrfdFaceDetection
modelPath:
thread: 4
name: ${VISUAL_MODEL_FACEDETECTION_NAME:InsightScrfdFaceDetection}
modelPath: ${VISUAL_MODEL_FACEDETECTION_PATH:}
thread: ${VISUAL_MODEL_FACEDETECTION_THREAD:4}
backup:
name: PcnNetworkFaceDetection
modelPath:
thread: 4
name: ${VISUAL_MODEL_FACEDETECTION_BACKUP_NAME:PcnNetworkFaceDetection}
modelPath: ${VISUAL_MODEL_FACEDETECTION_BACKUP_PATH:}
thread: ${VISUAL_MODEL_FACEDETECTION_BACKUP_THREAD:4}
faceKeyPoint:
name: InsightCoordFaceKeyPoint
modelPath:
thread: 4
name: ${VISUAL_MODEL_FACEKEYPOINT_NAME:InsightCoordFaceKeyPoint}
modelPath: ${VISUAL_MODEL_FACEKEYPOINT_PATH:}
thread: ${VISUAL_MODEL_FACEKEYPOINT_THREAD:4}
faceAlignment:
name: Simple005pFaceAlignment
modelPath:
thread: 4
name: ${VISUAL_MODEL_FACEALIGNMENT_NAME:Simple005pFaceAlignment}
modelPath: ${VISUAL_MODEL_FACEALIGNMENT_PATH:}
thread: ${VISUAL_MODEL_FACEALIGNMENT_THREAD:4}
faceRecognition:
name: InsightArcFaceRecognition
modelPath:
thread: 4
name: ${VISUAL_MODEL_FACERECOGNITION_NAME:InsightArcFaceRecognition}
modelPath: ${VISUAL_MODEL_FACERECOGNITION_PATH:}
thread: ${VISUAL_MODEL_FACERECOGNITION_THREAD:4}
faceAttribute:
name: InsightAttributeDetection
modelPath:
thread: 4
name: ${VISUAL_MODEL_FACEATTRIBUTE_NAME:InsightAttributeDetection}
modelPath: ${VISUAL_MODEL_FACEATTRIBUTE_PATH:}
thread: ${VISUAL_MODEL_FACEATTRIBUTE_THREAD:4}
engine:
selected: proxima
proxima:
host:
port: 16000
milvus:
host:
port: 19530
open-search:
host: ${VISUAL_ENGINE_OPENSEARCH_HOST}
port: ${VISUAL_ENGINE_OPENSEARCH_PORT:9200}
scheme: ${VISUAL_ENGINE_OPENSEARCH_SCHEME:https}
username: ${VISUAL_ENGINE_OPENSEARCH_USERNAME:admin}
password: ${VISUAL_ENGINE_OPENSEARCH_PASSWORD:admin}
scheduler:
flush:
enable: true
@ -62,7 +61,7 @@ visual:
face-search: false
face-compare: false
swagger:
enable: true
enable: ${VISUAL_SWAGGER_ENABLE:true}
# Spring配置
spring:
@ -83,9 +82,9 @@ spring:
druid:
# 主库数据源
master:
url:
username:
password:
url: ${SPRING_DATASOURCE_URL}
username: ${SPRING_DATASOURCE_USERNAME:root}
password: ${SPRING_DATASOURCE_PASSWORD:root}
slave:
# 从数据源开关/默认关闭
enabled: false

132
face-search-server/src/main/resources/application-local.yml Executable file
View File

@ -0,0 +1,132 @@
# 开发环境配置
server:
# 服务器的HTTP端口，默认为80
port: 8080
servlet:
# 应用的访问路径
context-path: /
tomcat:
# tomcat的URI编码
uri-encoding: UTF-8
# tomcat最大线程数，默认为200
max-threads: 10
# Tomcat启动初始化的线程数，默认值25
min-spare-threads: 5
# 日志配置
logging:
level:
com.visual.face.search: info
org.springframework: warn
# 模型配置
visual:
model:
faceDetection:
name: InsightScrfdFaceDetection
modelPath:
thread: 1
backup:
name: PcnNetworkFaceDetection
modelPath:
thread: 1
faceKeyPoint:
name: InsightCoordFaceKeyPoint
modelPath:
thread: 1
faceAlignment:
name: Simple005pFaceAlignment
modelPath:
thread: 1
faceRecognition:
name: InsightArcFaceRecognition
modelPath:
thread: 1
faceAttribute:
name: InsightAttributeDetection
modelPath:
thread: 1
engine:
open-search:
host: 172.16.36.229
port: 9200
scheme: https
username: admin
password: admin
scheduler:
flush:
enable: true
interval: 60000
face-mask:
face-search: false
face-compare: false
swagger:
enable: true
# Spring配置
spring:
jackson:
time-zone: GMT+8
date-format: yyyy-MM-dd HH:mm:ss
# 文件上传
servlet:
multipart:
# 单个文件大小
max-file-size: 10MB
# 设置总上传的文件大小
max-request-size: 20MB
#数据源
datasource:
type: com.alibaba.druid.pool.DruidDataSource
driverClassName: com.mysql.cj.jdbc.Driver
druid:
# 主库数据源
master:
url: jdbc:mysql://172.16.36.228:3306/visual_face_search?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&useSSL=false&serverTimezone=GMT%2B8
username: visual
password: visual
slave:
# 从数据源开关/默认关闭
enabled: false
url:
username:
password:
# 初始连接数
initialSize: 5
# 最小连接池数量
minIdle: 10
# 最大连接池数量
maxActive: 20
# 配置获取连接等待超时的时间
maxWait: 60000
# 配置间隔多久才进行一次检测，检测需要关闭的空闲连接，单位是毫秒
timeBetweenEvictionRunsMillis: 60000
# 配置一个连接在池中最小生存的时间，单位是毫秒
minEvictableIdleTimeMillis: 300000
# 配置一个连接在池中最大生存的时间，单位是毫秒
maxEvictableIdleTimeMillis: 900000
# 配置检测连接是否有效
validationQuery: SELECT 1 FROM DUAL
testWhileIdle: true
testOnBorrow: false
testOnReturn: false
webStatFilter:
enabled: true
statViewServlet:
enabled: true
# 设置白名单，不填则允许所有访问
allow:
url-pattern: /druid/*
# 控制台管理用户名和密码
login-username:
login-password:
filter:
stat:
enabled: true
# 慢SQL记录
log-slow-sql: true
slow-sql-millis: 1000
merge-sql: true
wall:
config:
multi-statement-allow: true

5
face-search-server/src/main/resources/application.yml
View File

@ -1,5 +1,8 @@
spring:
application:
name: open-face-search
mvc:
pathmatch:
matching-strategy: ant_path_matcher
profiles:
active: dev
active: local

6
face-search-test/pom.xml
View File

@ -4,9 +4,9 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>face-search-test</artifactId>
<groupId>com.visual.face.search</groupId>
<version>1.2.0</version>
<artifactId>face-search-test</artifactId>
<version>2.0.0</version>
<properties>
<java.version>1.8</java.version>
@ -18,7 +18,7 @@
<dependency>
<groupId>com.visual.face.search</groupId>
<artifactId>face-search-client</artifactId>
<version>1.1.0</version>
<version>${project.version}</version>
</dependency>
<dependency>

33
face-search-test/src/main/java/com/visual/face/search/valid/exps/FaceSearchExample.java
View File

@ -23,7 +23,7 @@ public class FaceSearchExample {
//远程测试服务
//public static String serverHost = "http://face-search.diven.nat300.top";
public static String namespace = "namespace_1";
public static String collectionName = "collect_20211201_v05";
public static String collectionName = "collect_20211201_v10";
public static FaceSearch faceSearch = FaceSearch.build(serverHost, namespace, collectionName);
/**集合创建*/
@ -67,8 +67,8 @@ public class FaceSearchExample {
faceData.add(KeyValue.build("label", "标签-" + name));
String imageBase64 = Base64Util.encode(image.getAbsolutePath());
Face face = Face.build(sampleId).setFaceData(faceData).setImageBase64(imageBase64)
.setMinConfidenceThresholdWithThisSample(0f)
.setMaxConfidenceThresholdWithOtherSample(0f);
.setMinConfidenceThresholdWithThisSample(50f)
.setMaxConfidenceThresholdWithOtherSample(50f);
Response<FaceRep> createFace = faceSearch.face().createFace(face);
System.out.println("createFace:" + createFace);
}
@ -82,8 +82,13 @@ public class FaceSearchExample {
String searchPath = "face-search-test/src/main/resources/image/validate/search";
for(File image : Objects.requireNonNull(new File(searchPath).listFiles())){
String imageBase64 = Base64Util.encode(image.getAbsolutePath());
System.out.println(imageBase64);
Long s = System.currentTimeMillis();
Response<List<SearchRep>> listResponse = faceSearch.search().search(Search.build(imageBase64).setMaxFaceNum(10).setLimit(1));
Response<List<SearchRep>> listResponse = faceSearch.search()
.search(Search.build(imageBase64)
.setConfidenceThreshold(50f) //最小置信分：50
.setMaxFaceNum(10).setLimit(1)
);
Long e = System.currentTimeMillis();
System.out.println("search cost:" + (e-s)+"ms");
System.out.println(JsonUtil.toString(listResponse, true, false));
@ -95,13 +100,15 @@ public class FaceSearchExample {
FaceLocation location = rep.getLocation();
drawImage.drawRect(new DrawImage.Rect(location.getX(), location.getY(), location.getW(), location.getH()), 2, Color.RED);
List<SampleFace> faces = rep.getMatch();
for(SampleFace face : faces){
int distance = face.getDistance().intValue();
int confidence = face.getConfidence().intValue();
String name = face.getSampleData().getString("name");
drawImage.drawText("姓名：" + name, new DrawImage.Point(location.getX() + 5, location.getY()+5), 14, Color.RED);
drawImage.drawText("分数：" + confidence, new DrawImage.Point(location.getX()+5, location.getY()+25), 14, Color.RED);
drawImage.drawText("距离：" + distance, new DrawImage.Point(location.getX()+5, location.getY()+50), 14, Color.RED);
if(faces.size() > 0){
for(SampleFace face : faces){
int confidence = face.getConfidence().intValue();
String name = face.getSampleData().getString("name");
drawImage.drawText("姓名：" + name, new DrawImage.Point(location.getX() + 5, location.getY()+5), 14, Color.RED);
drawImage.drawText("分数：" + confidence, new DrawImage.Point(location.getX()+5, location.getY()+25), 14, Color.RED);
}
}else{
drawImage.drawText("未识别", new DrawImage.Point(location.getX() + 5, location.getY()+5), 14, Color.GREEN);
}
}
}
@ -115,8 +122,8 @@ public class FaceSearchExample {
/**main**/
public static void main(String[] args) {
// collect();
// index();
collect();
index();
search();
}

62
pom.xml
View File

@ -3,39 +3,44 @@
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.1.RELEASE</version>
<version>2.7.5</version>
<relativePath/>
</parent>
<groupId>com.visual.face.search</groupId>
<artifactId>face-search</artifactId>
<version>1.2.0</version>
<packaging>pom</packaging>
<version>2.0.0</version>
<modules>
<module>face-search-core</module>
<module>face-search-server</module>
<module>face-search-client</module>
<module>face-search-engine</module>
<module>face-search-server</module>
<module>face-search-test</module>
</modules>
<properties>
<druid.version>1.1.22</druid.version>
<opencv.version>4.5.1-2</opencv.version>
<onnxruntime.version>1.9.0</onnxruntime.version>
<opensearch.version>2.4.0</opensearch.version>
<onnxruntime.version>1.13.1</onnxruntime.version>
<fastjson.version>1.2.58</fastjson.version>
<hibernate.version>6.0.13.Final</hibernate.version>
<commons-math3.version>3.6.1</commons-math3.version>
<commons-collections4.version>4.1</commons-collections4.version>
<mybatis.version>3.4.6</mybatis.version>
<mybatis-spring.version>1.3.1</mybatis-spring.version>
<pagehelper.version>5.1.4</pagehelper.version>
<pagehelper-starter.version>1.2.3</pagehelper-starter.version>
<swagger.version>2.9.2</swagger.version>
<swagger-bootstrap-ui.version>1.9.6</swagger-bootstrap-ui.version>
<java.version>1.8</java.version>
<pagehelper-starter.version>1.4.5</pagehelper-starter.version>
<swagger.version>3.0.0</swagger.version>
<knife4j-ui.version>3.0.3</knife4j-ui.version>
<java.version>11</java.version>
<maven-resources-plugin.version>2.6</maven-resources-plugin.version>
</properties>
@ -71,6 +76,12 @@
<version>${onnxruntime.version}</version>
</dependency>
<dependency>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-rest-high-level-client</artifactId>
<version>${opensearch.version}</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
@ -96,6 +107,12 @@
<version>${druid.version}</version>
</dependency>
<dependency>
<groupId>org.hibernate.validator</groupId>
<artifactId>hibernate-validator</artifactId>
<version>${hibernate.version}</version>
</dependency>
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
@ -121,15 +138,14 @@
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<artifactId>springfox-boot-starter</artifactId>
<version>${swagger.version}</version>
</dependency>
<dependency>
<groupId>com.github.xiaoymin</groupId>
<artifactId>swagger-bootstrap-ui</artifactId>
<version>${swagger-bootstrap-ui.version}</version>
<artifactId>knife4j-spring-boot-starter</artifactId>
<version>${knife4j-ui.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
@ -144,6 +160,7 @@
<excludes>
<exclude>model/**</exclude>
<exclude>application-dev.yml</exclude>
<exclude>application-local.yml</exclude>
</excludes>
</resource>
</resources>
@ -160,6 +177,14 @@
<enabled>true</enabled>
</releases>
</repository>
<repository>
<id>central</id>
<name>aliyun nexus</name>
<url>https://maven.aliyun.com/repository/central</url>
<releases>
<enabled>true</enabled>
</releases>
</repository>
</repositories>
<pluginRepositories>
@ -170,4 +195,17 @@
</pluginRepository>
</pluginRepositories>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>11</source>
<target>11</target>
</configuration>
</plugin>
</plugins>
</build>
</project>

96
scripts/docker-compose-milvus.yml
View File

@ -1,96 +0,0 @@
version: '3.5'
services:
visual-mysql:
container_name: face-search-server-mysql
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: visual_face_search
MYSQL_ROOT_PASSWORD: root
MYSQL_ROOT_HOST: '%'
TZ: Asia/Shanghai
expose:
- "3306"
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/mysql:/var/lib/mysql
ports:
- "43306:3306"
restart: always
visual-etcd:
container_name: face-search-milvus-etcd
image: quay.io/coreos/etcd:v3.5.0
environment:
- ETCD_AUTO_COMPACTION_MODE=revision
- ETCD_AUTO_COMPACTION_RETENTION=1000
- ETCD_QUOTA_BACKEND_BYTES=4294967296
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/etcd:/etcd
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
restart: always
visual-minio:
container_name: face-search-milvus-minio
image: minio/minio:RELEASE.2020-12-03T00-03-10Z
environment:
MINIO_ACCESS_KEY: minioadmin
MINIO_SECRET_KEY: minioadmin
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/minio:/minio_data
command: minio server /minio_data
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
interval: 30s
timeout: 20s
retries: 3
restart: always
visual-milvus:
container_name: face-search-milvus-standalone
image: milvusdb/milvus:v2.0.2
command: ["milvus", "run", "standalone"]
environment:
ETCD_ENDPOINTS: visual-etcd:2379
MINIO_ADDRESS: visual-minio:9000
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/milvus:/var/lib/milvus
ports:
- "19530:19530"
depends_on:
- "visual-etcd"
- "visual-minio"
restart: always
visual-facesearch:
container_name: face-search-server-standalone
image: divenswu/face-search:1.2.0
environment:
SPRING_DATASOURCE_URL: 'jdbc:mysql://visual-mysql:3306/visual_face_search?useUnicode=true&characterEncoding=utf8'
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
VISUAL_ENGINE_SELECTED: milvus
VISUAL_ENGINE_MILVUS_HOST: visual-milvus
VISUAL_ENGINE_MILVUS_PORT: 19530
VISUAL_SWAGGER_ENABLE: 'true'
JAVA_OPTS: '-XX:MaxDirectMemorySize=400M'
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/search/logs:/app/face-search/logs
ports:
- "56789:8080"
deploy:
resources:
limits:
cpus: '2'
memory: 1G
reservations:
cpus: '1'
memory: 500M
depends_on:
- "visual-mysql"
- "visual-milvus"
restart: always
networks:
default:
name: facesearch-milvus

64
scripts/docker-compose-opensearch.yml Normal file
View File

@ -0,0 +1,64 @@
version: '3.5'
services:
visual-mysql:
container_name: face-search-server-mysql
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: visual_face_search
MYSQL_ROOT_PASSWORD: root
MYSQL_ROOT_HOST: '%'
TZ: Asia/Shanghai
expose:
- "3306"
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
volumes:
- ${FACESEARCH_VOLUME_DIRECTORY:-.}/volumes-face-search/mysql:/var/lib/mysql
restart: always
visual-opensearch:
container_name: face-search-opensearch-standalone
image: opensearchproject/opensearch:2.4.0
environment:
discovery.type: single-node
expose:
- "9200"
- "9600"
volumes:
- ${FACESEARCH_VOLUME_DIRECTORY:-.}/volumes-face-search/opensearch/data:/usr/share/opensearch/data
restart: always
visual-opensearch-dashboards:
container_name: face-search-opensearch-dashboards
image: opensearchproject/opensearch-dashboards:2.4.0
environment:
OPENSEARCH_HOSTS: '["https://visual-opensearch:9200"]'
ports:
- "5601:5601"
depends_on:
- "visual-opensearch"
restart: always
visual-facesearch:
container_name: face-search-server-standalone
image: divenswu/face-search:2.0.0
environment:
SPRING_DATASOURCE_URL: 'jdbc:mysql://visual-mysql:3306/visual_face_search?useUnicode=true&characterEncoding=utf8'
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
VISUAL_ENGINE_OPENSEARCH_HOST: visual-opensearch
VISUAL_ENGINE_OPENSEARCH_PORT: 9200
VISUAL_SWAGGER_ENABLE: 'true'
JAVA_OPTS: '-XX:MaxDirectMemorySize=400M'
volumes:
- ${FACESEARCH_VOLUME_DIRECTORY:-.}/volumes-face-search/search/logs:/app/face-search/logs
ports:
- "56789:8080"
depends_on:
- "visual-mysql"
- "visual-opensearch"
restart: always
networks:
default:
name: facesearch-opensearch

62
scripts/docker-compose-proxima.yml
View File

@ -1,62 +0,0 @@
version: '3.5'
services:
visual-mysql:
container_name: face-search-server-mysql
image: mysql/mysql-server:5.7
environment:
MYSQL_DATABASE: visual_face_search
MYSQL_ROOT_PASSWORD: root
MYSQL_ROOT_HOST: '%'
TZ: Asia/Shanghai
expose:
- "3306"
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/mysql:/var/lib/mysql
restart: always
visual-proxima:
container_name: face-search-proxima-standalone
image: divenswu/proxima-be:0.2.0-official
# image: ghcr.io/proximabilin/proxima-be:0.2.0 # so slowly
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/proxima/data:/var/lib/proxima-be/data
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/proxima/log:/var/lib/proxima-be/log
# ports:
# - "16000:16000" #grpc
# - "16001:16001" #http
restart: always
visual-facesearch:
container_name: face-search-server-standalone
image: divenswu/face-search:1.2.0
environment:
SPRING_DATASOURCE_URL: 'jdbc:mysql://visual-mysql:3306/visual_face_search?useUnicode=true&characterEncoding=utf8'
SPRING_DATASOURCE_USERNAME: root
SPRING_DATASOURCE_PASSWORD: root
VISUAL_ENGINE_SELECTED: proxima
VISUAL_ENGINE_PROXIMA_HOST: visual-proxima
VISUAL_ENGINE_PROXIMA_PORT: 16000
VISUAL_SWAGGER_ENABLE: 'true'
JAVA_OPTS: '-XX:MaxDirectMemorySize=400M'
volumes:
- ${DOCKER_VOLUME_DIRECTORY:-.}/volumes-face-search/search/logs:/app/face-search/logs
ports:
- "56789:8080"
deploy:
resources:
limits:
cpus: '2'
memory: 1G
reservations:
cpus: '1'
memory: 500M
depends_on:
- "visual-mysql"
- "visual-proxima"
restart: always
networks:
default:
name: facesearch-proxima

6
scripts/docker/Dockerfile
View File

@ -1,9 +1,9 @@
FROM frolvlad/alpine-oraclejre8
FROM openjdk:11.0.11-jre-slim
MAINTAINER face-search
WORKDIR /app/face-search
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && \
apk update && apk add libssl1.0 libx11 libxext libxrender libstdc++ freetype fontconfig
#RUN sed -i 's/deb.debian.org/mirrors.aliyun.com/g' /etc/apt/sources.list && \
#apk update && apk add libssl1.0 libx11 libxext libxrender libstdc++ freetype fontconfig
COPY scripts/docker/entrypoint.sh /app/face-search
RUN chmod +x /app/face-search/entrypoint.sh

96
scripts/docker/entrypoint.sh
View File

@ -11,100 +11,4 @@ else
fi
################################################## active config end ###################################################
################################################## swagger config start ################################################
if [ "${VISUAL_SWAGGER_ENABLE}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.swagger.enable='$VISUAL_SWAGGER_ENABLE'"
fi
################################################## swagger config end ##################################################
################################################## datasource config start #############################################
if [ "${SPRING_DATASOURCE_URL}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dspring.datasource.druid.master.url='$SPRING_DATASOURCE_URL'"
fi
if [ "${SPRING_DATASOURCE_USERNAME}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dspring.datasource.druid.master.username='$SPRING_DATASOURCE_USERNAME'"
fi
if [ "${SPRING_DATASOURCE_PASSWORD}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dspring.datasource.druid.master.password='$SPRING_DATASOURCE_PASSWORD'"
fi
################################################## datasource config end ###############################################
###################################################### engine config start #############################################
if [ "${VISUAL_ENGINE_SELECTED}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.engine.selected='$VISUAL_ENGINE_SELECTED'"
fi
if [ "${VISUAL_ENGINE_PROXIMA_HOST}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.engine.proxima.host='$VISUAL_ENGINE_PROXIMA_HOST'"
fi
if [ "${VISUAL_ENGINE_PROXIMA_PORT}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.engine.proxima.port='$VISUAL_ENGINE_PROXIMA_PORT'"
fi
if [ "${VISUAL_ENGINE_MILVUS_HOST}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.engine.milvus.host='$VISUAL_ENGINE_MILVUS_HOST'"
fi
if [ "${VISUAL_ENGINE_MILVUS_PORT}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.engine.milvus.port='$VISUAL_ENGINE_MILVUS_PORT'"
fi
###################################################### engine config end ###############################################
###################################################### model config start ##############################################
if [ "${VISUAL_MODEL_FACEDETECTION_NAME}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceDetection.name='$VISUAL_MODEL_FACEDETECTION_NAME'"
fi
if [ "${VISUAL_MODEL_FACEDETECTION_PATH}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceDetection.modelPath='$VISUAL_MODEL_FACEDETECTION_PATH'"
fi
if [ "${VISUAL_MODEL_FACEDETECTION_THREAD}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceDetection.thread='$VISUAL_MODEL_FACEDETECTION_THREAD'"
fi
if [ "${VISUAL_MODEL_FACEDETECTION_BACKUP_NAME}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceDetection.backup.name='$VISUAL_MODEL_FACEDETECTION_BACKUP_NAME'"
fi
if [ "${VISUAL_MODEL_FACEDETECTION_BACKUP_PATH}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceDetection.backup.modelPath='$VISUAL_MODEL_FACEDETECTION_BACKUP_PATH'"
fi
if [ "${VISUAL_MODEL_FACEDETECTION_BACKUP_THREAD}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceDetection.backup.thread='$VISUAL_MODEL_FACEDETECTION_BACKUP_THREAD'"
fi
if [ "${VISUAL_MODEL_FACEKEYPOINT_NAME}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceKeyPoint.name='$VISUAL_MODEL_FACEKEYPOINT_NAME'"
fi
if [ "${VISUAL_MODEL_FACEKEYPOINT_PATH}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceKeyPoint.modelPath='$VISUAL_MODEL_FACEKEYPOINT_PATH'"
fi
if [ "${VISUAL_MODEL_FACEKEYPOINT_THREAD}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceKeyPoint.thread='$VISUAL_MODEL_FACEKEYPOINT_THREAD'"
fi
if [ "${VISUAL_MODEL_FACEALIGNMENT_NAME}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceAlignment.name='$VISUAL_MODEL_FACEALIGNMENT_NAME'"
fi
if [ "${VISUAL_MODEL_FACEALIGNMENT_PATH}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceAlignment.modelPath='$VISUAL_MODEL_FACEALIGNMENT_PATH'"
fi
if [ "${VISUAL_MODEL_FACEALIGNMENT_THREAD}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceAlignment.thread='$VISUAL_MODEL_FACEALIGNMENT_THREAD'"
fi
if [ "${VISUAL_MODEL_FACERECOGNITION_NAME}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceRecognition.name='$VISUAL_MODEL_FACERECOGNITION_NAME'"
fi
if [ "${VISUAL_MODEL_FACERECOGNITION_PATH}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceRecognition.modelPath='$VISUAL_MODEL_FACERECOGNITION_PATH'"
fi
if [ "${VISUAL_MODEL_FACERECOGNITION_THREAD}" ];then
SPRING_PROFILE_CONFIG="${SPRING_PROFILE_CONFIG} -Dvisual.model.faceRecognition.thread='$VISUAL_MODEL_FACERECOGNITION_THREAD'"
fi
###################################################### model config end ###############################################
sh -c "java -server ${SPRING_PROFILE_CONFIG} ${SPRING_OPTS} ${JAVA_OPTS} -jar /app/face-search/face-search-server.jar"

2
scripts/docker_build.sh
View File

@ -1,4 +1,4 @@
version='1.2.0'
version='2.0.0'
SHELL_FOLDER=$(cd "$(dirname "$0")";pwd)
cd ${SHELL_FOLDER}

1210
scripts/docs/2.0.0.md Normal file
View File

File diff suppressed because it is too large Load Diff

1230
scripts/docs/doc-1.0.0.md
View File

File diff suppressed because it is too large Load Diff

1340
scripts/docs/doc-1.1.0.md
View File

File diff suppressed because it is too large Load Diff

BIN
scripts/images/validate-1.2.0.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 139 KiB

BIN
scripts/images/validate-2.0.0.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 134 KiB

BIN
scripts/images/validate.jpg
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 136 KiB