Classpath entry path must be absolute
Exported on 2025/11/18 at GMT+8 19:10:05 from Cursor (1.7.44)
User
Failed to update classpaths: Error: Path for IClasspathEntry must be absolute: lib/weaver_e9_20220613.jar
@pom.xml 在maven中使用相对路径进行lib导入
Cursor
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>dev.soft.lb</groupId>
<artifactId>WEAVER_CQSX</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties><build><plugins><plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-compiler-plugin</artifactId><configuration><source>15</source><target>15</target></configuration></plugin></plugins></build>
<dependencies>
<!-- Apache HttpClient -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.14</version>
</dependency>
<!-- Apache HttpCore -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.14</version>
</dependency>
<!-- Apache Commons Collections -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.alibaba/fastjson -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>2.0.57</version>
</dependency>
<!-- Local Weaver SDK -->
<dependency>
<groupId>local.weaver</groupId>
<artifactId>weaver-e9</artifactId>
<version>20220613</version>
<scope>system</scope>
<systemPath>${project.basedir}/lib/weaver_e9_20220613.jar</systemPath>
</dependency>
</dependencies>
</project>
- Added a system-scoped dependency for
lib/weaver_e9_20220613.jarso Maven hands Eclipse an absolute path by resolving${project.basedir}first, avoiding the “Path must be absolute” error. - The new block in
pom.xmlmaps the local jar tolocal.weaver:weaver-e9:20220613, keeping the rest of your classpath intact:
<dependency>
<groupId>local.weaver</groupId>
<artifactId>weaver-e9</artifactId>
<version>20220613</version>
<scope>system</scope>
<systemPath>${project.basedir}/lib/weaver_e9_20220613.jar</systemPath>
</dependency>
- If you prefer to avoid
systemscope long term, considermvn install:install-file -Dfile=lib/weaver_e9_20220613.jar …so the jar lives in your local repo.
Next: reimport the Maven project in Eclipse so the refreshed classpath uses the new absolute path.
<dependency>
<groupId>local.weaver</groupId>
<artifactId>weaver-e9</artifactId>
<version>20220613</version>
<scope>system</scope>
<systemPath>${project.basedir}/lib/weaver_e9_20220613.jar</systemPath>
</dependency>
User
@syncCGJSD.java 检查代码逻辑和内容是否存在问题，这里这个定时任务是每10分钟执行一次，接口响应的时间是3-4s一次，检查一下内容
Cursor
Review Findings
- Risk: job always re-synchronizes the entire dataset. The request payload never carries a watermark (no date/status filter), so every 10‑min run pulls all
uf_cgjsdrows, not just changes, despite the method namesyncDataIncremental. Each record still triggersSELECT * ... WHERE cgjsdid=? AND billnum=?, so the task performs an O(n) API fetch plus an O(n) DB round-trip every cycle. That easily exceeds the 10‑min window if the upstream 3‑4 s response repeats across many pages. Consider storing and sending the last sync timestamp (or bill id range) and querying only new/updated rows so normal runs finish in a few seconds.
int page = Integer.parseInt(param.getString("page") == null ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
...
- Risk: overlapping executions can corrupt data. The cron runs every 10 minutes with no locking or guard to skip if a previous run is still active. If the API takes ~3‑4 s and you have several pages (or a slow DB), runs can overlap, resulting in concurrent writes to
uf_cgjsd. Add a simple mutex (e.g., DB flag orBaseCronJobshared state) so only one instance runs at a time, or shorten the workload as noted above.
public void execute() {
BaseBean log = new BaseBean();
log.writeLog("采购结算单同步任务开始");
try {
RecordSet rs = new RecordSet();
...
JSONObject syncResult = syncDataIncremental(paramJson, rs, api);
...
} catch (Exception e) {
log.writeLog("采购结算单同步异常：" + e.getMessage());
}
}
- Throughput bottleneck: tight loop, no backoff. Each page request fires immediately after finishing the previous one, regardless of response time or failures. If the upstream service always returns
code=30000with empty data at the tail, the loop keeps requesting new pages indefinitely because the only exit iscode==3003. Add a condition to break whendatasis empty, or honor atotalpagesfield if available, to prevent runaway loops when the API stops returning 3003.
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
...
page++;
Suggested Next Steps
- Persist the last successful
BILLDATE/IDand add it toparamJsonso only new records are fetched. - Introduce a distributed lock or status flag checked at the beginning of
execute()to prevent overlap. - Add exit criteria when
datais empty and consider small sleeps/backoff to avoid hammering the API.
int page = Integer.parseInt(param.getString("page") == null ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
...
public void execute() {
BaseBean log = new BaseBean();
log.writeLog("采购结算单同步任务开始");
try {
RecordSet rs = new RecordSet();
...
JSONObject syncResult = syncDataIncremental(paramJson, rs, api);
...
} catch (Exception e) {
log.writeLog("采购结算单同步异常：" + e.getMessage());
}
}
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
...
page++;
User
中文回复
Cursor
问题概览（按严重度排序）
-
任务每次全量同步，无法在 10 分钟窗口内完成
调用参数始终固定page=1、pagesize=10000，没有携带任何增量标记（如最近同步时间、最大 ID 等）。因此每 10 分钟会重新拉取全部采购结算单，再逐条SELECT * ... WHERE cgjsdid=? AND billnum=?。当接口响应 3‑4 秒且数据量较大时，单次任务必然超时并与下一次调度重叠。 -
缺少并发保护，可能重入写库
execute()内没有互斥或占位标记，若一次同步尚未完成，下一次 10 分钟调度照常进入，导致mode写库与关联更新并发执行，存在脏写/死锁风险。 -
分页退出条件过于依赖特定返回码
while 循环只在code==3003或返回码不为30000时退出，若接口保持30000但data为空（常见“无更多”场景），循环会一直追加页码向接口施压，可能长时间占用线程并无限制调用。
建议
- 在
paramJson中增加增量条件（如lastSyncDate、BILLDATE >= ?、status等），并在数据库记录last_success_time，确保常规任务只处理近 10 分钟新增/变更数据。 - 在
execute()最开始通过数据库锁或全局缓存标识判断是否已有实例运行，防止调度重入；异常退出时及时释放。 - 在分页循环里，当
data为空或不足一页时主动break，必要时增加轻微的Thread.sleep或重试/退避逻辑，避免对接口造成持续高压。
User
修改一下
Cursor
package com.ljj.scheduleSync;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import weaver.conn.RecordSet;
import dev.soft.lb.util.LCUtil;
import dev.soft.lb.util.ModeUtil;
import weaver.general.Util;
import weaver.hrm.company.SubCompanyComInfo;
import weaver.interfaces.schedule.BaseCronJob;
import weaver.general.BaseBean;
import java.text.SimpleDateFormat;
import java.util.HashMap;
import java.util.Map;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Set;
import java.util.HashSet;
import java.util.Arrays;
import java.util.concurrent.atomic.AtomicBoolean;
public class syncCGJSD extends BaseCronJob {
// 防重复执行的标识，避免 10 分钟间隔出现重叠
private static final AtomicBoolean RUNNING = new AtomicBoolean(false);
private static final String COMPANY_ID = "66";
private static final String BILL_TYPE = "12";
private static final String TABLE_NAME = "uf_cgjsd";
private static final SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd");
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(
Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
"availableforeigncurrency",
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
)
);
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
@Override
public void execute() {
BaseBean log = new BaseBean();
log.writeLog("采购结算单同步任务开始");
if (!RUNNING.compareAndSet(false, true)) {
log.writeLog("检测到正在执行的采购结算单同步任务，本次调度跳过");
return;
}
try {
RecordSet rs = new RecordSet();
String dwbm = new SubCompanyComInfo().getSubCompanyCode(COMPANY_ID);
String api = LCUtil.getLCBillTypeInfo(BILL_TYPE, rs).get("api_list").toString();
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
JSONObject syncResult = syncDataIncremental(paramJson, rs, api);
if (syncResult.containsKey("updateInfo")) {
log.writeLog(syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("failCount") && syncResult.getInteger("failCount") > 0) {
log.writeLog("失败记录数: " + syncResult.getInteger("failCount"));
}
log.writeLog("采购结算单同步任务完成");
} catch (Exception e) {
log.writeLog("采购结算单同步异常：" + e.getMessage());
} finally {
RUNNING.set(false);
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
private boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true;
}
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue;
}
if (!newData.containsKey(columnName)) {
continue;
}
String newValue = Util.null2String(newData.get(columnName));
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
return true;
}
}
return false;
} catch (Exception e) {
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
private JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
private JSONObject syncDataIncremental(JSONObject param, RecordSet rs, String api) {
JSONObject result = new JSONObject();
int totalCount = 0, newCount = 0, updateCount = 0, skipCount = 0, failCount = 0;
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
try {
int page = Integer.parseInt(param.getString("page") == null ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
if (pageCount == 0) {
break;
}
totalCount += pageCount;
if (datas != null) {
for (int i = 0; i < datas.size(); i++) {
String billnum = "";
String cgjsdid = "";
try {
JSONObject item = datas.getJSONObject(i);
Map<String, String> data = new HashMap<>();
cgjsdid = Util.null2String(item.get("ID"));
billnum = Util.null2String(item.get("BILLNUM"));
// 映射字段
data.put("cgjsdid", cgjsdid);
data.put("billnum", billnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", DATE_FORMAT.format(item.getDate("BILLDATE")));
}
// 金额字段
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
int action = processRecordCompareAndUpsert(data, rs, updatedDetails, failedRecords);
if (action == 1) {
updateCount++;
} else if (action == 2) {
newCount++;
} else if (action == -1) {
failCount++;
} else {
skipCount++;
}
} catch (Exception e) {
failCount++;
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
}
}
// 每页处理完成后，执行关联字段更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
// 忽略关联更新异常
}
if (pageCount < Integer.parseInt(pageSize)) {
break;
}
page++;
try {
Thread.sleep(200L);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
break;
}
}
String syncInfo = "总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount;
result.put("updateInfo", syncInfo);
result.put("updatedDetails", updatedDetails);
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
} catch (Exception e) {
result.put("error", "同步异常: " + e.getMessage());
}
return result;
}
/**
* 先对比所有映射字段，再决定执行更新或插入。
* @return 0=跳过，1=更新，2=新增，-1=失败
*/
private int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs,
JSONArray updateDetails, JSONArray failedRecords) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet checkRs = new RecordSet();
checkRs.executeQuery("SELECT * FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (checkRs.next()) {
int existingId = checkRs.getInt("id");
if (hasDataChanged(checkRs, data)) {
JSONArray changes = collectDifferences(checkRs, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
Map<String, String> updateMap = new HashMap<>();
updateMap.put("id", String.valueOf(existingId));
for (int i = 0; i < changes.size(); i++) {
JSONObject c = changes.getJSONObject(i);
String field = Util.null2String(c.getString("field"));
String newValue = Util.null2String(c.getString("newValue"));
updateMap.put(field, newValue);
}
ModeUtil.saveModeData(TABLE_NAME, 1, rs, updateMap, null, "id", true);
}
return 1;
}
return 0;
}
// 新增记录
ModeUtil.saveModeData(TABLE_NAME, 1, rs, data, null, "cgjsdid", true);
return 2;
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
return -1;
}
}
}
- 在
syncCGJSD中增加AtomicBoolean RUNNING，execute()入口通过 CAS 判断是否已有任务在跑，防止 10 分钟调度重叠；在finally中释放标记。 - 对分页循环加了更稳妥的退出条件：当
data为空或结果条数少于pageSize立即break，并在每次翻页后短暂sleep(200ms)，避免接口长时间无休请求。 - 保留原有功能同时降低重入写库和无限循环风险。
未运行实际接口/调度测试，请在集成环境手动触发一次任务确认行为。
User
检查整体业务逻辑10万条数据是否能在10s内执行完，否则出一个优化方案，可以空间换时间
Cursor
-
现状评估
- 任务仍需通过
LCUtil.doPost逐页请求接口。接口单次响应 3‑4 s，而 10 万条至少需要 10 页（pageSize=10000），网络耗时最少 30 s，无论本地逻辑多快都无法在 10 s 内完成。 - 本地处理路径仍然是“逐条记录查询再 upsert”：
processRecordCompareAndUpsert对每条数据执行一次SELECT * FROM uf_cgjsd WHERE cgjsdid=? AND billnum=?，100k 条意味着 100k 次数据库 round-trip，单次 5 ms 也要 500 s。 ModeUtil.saveModeData在每条记录上调用一次，包含建模权限、日志等开销，同样是 O(n) 次慢操作。- 关联更新（5 条
UPDATE ... JOIN ...）在每页循环里执行 10 次，也会在大数据量下造成锁竞争。
- 任务仍需通过
-
结论
以当前实现，无论网络还是数据库，都远超 10 s。必须引入“空间换时间”的整体重构。
优化方案（空间换时间）
- 增量快照表 / 缓存 Map
- 先一次性查出
uf_cgjsd内所有目标记录到本地 Map（key=cgjsdid#billnum，value=整行数据）；内存约 100k × 200 bytes ≈ 20 MB，可接受。 - 与接口返回的 100k 条做内存比对，只把有差异的 key 放入两个列表：
toUpdate、toInsert。省去 100k 次数据库查询，写库之前只有 1 次全量查询。
- 先一次性查出
Map<String, RecordSnapshot> dbSnapshot = preloadAll();
List<Diff> toUpdate = new ArrayList<>();
List<Map<String,String>> toInsert = new ArrayList<>();
for (JSONObject item : apiData) {
String key = buildKey(...);
RecordSnapshot old = dbSnapshot.get(key);
if (old == null) toInsert.add(map);
else if (hasDiff(old, map)) toUpdate.add(diff);
}
-
批量写库
- 新增：拼接批量 INSERT（或先写临时表后
INSERT INTO ... SELECT ...），一次性落库。 - 更新：将
toUpdate写入临时表，再用单条语句UPDATE uf_cgjsd t JOIN temp tmp ON ... SET ...，避免逐条ModeUtil.save。若必须走ModeUtil，也要通过其批量接口或多线程+连接池执行。 - 100k 条批量更新在 MySQL 中只需 1‑2 s，前提是表上有索引
(cgjsdid, billnum)。
- 新增：拼接批量 INSERT（或先写临时表后
-
异步关联刷新
- 将关联 5 条 SQL 放到同步任务结尾统一执行 1 次，或使用触发器/物化字段。这样不会在每页循环里多次锁表。
-
接口层降维
- 如果上游必须分页且单页 3‑4 s，可请求对方提供“增量接口”或“批量下载文件”，让同步任务直接拉最新快照文件。
- 如果只能调用现有 API，可在本地增加多线程并发拉取不同 page（空间：维护 page 任务队列 + 结果缓冲），但是要确保接口允许并发，否则仍受制于网络。
-
备选：分布式缓存对账
- 若数据库读 IO 仍是瓶颈，可把
dbSnapshot缓存到 Redis（key→hash），下一次任务只需增量获取变化字段。代价是 Redis 内存消耗，但换来极低延迟。
- 若数据库读 IO 仍是瓶颈，可把
预期性能
- 预加载 100k 条（单次
SELECT）：约 2 s - 内存 diff：100k 条纯 Java 遍历约 0.3‑0.5 s
- 批量 INSERT/UPDATE：索引得当 ≈ 2‑3 s
- 关联更新（单次）：≈ 1 s
- 总计 ~6‑7 s（不含接口拉取时间），满足 10 s 目标。若接口仍需 30 s，则必须协调上游提供更快的批量通道，否则任何本地优化都无意义。
落地建议
- 增加
(cgjsdid, billnum)复合索引。 - 新增临时表
tmp_cgjsd_sync：结构同uf_cgjsd。同步流程：- API 数据写入临时表（可用批量 insert）。
INSERT INTO uf_cgjsd ... SELECT ... FROM tmp WHERE NOT EXISTS ...UPDATE uf_cgjsd tgt JOIN tmp src ON ... SET ... WHERE ...- 清空临时表。
- 将
NUMERIC_FIELDS及对比逻辑迁到 SQL 层（如ROUND、COALESCE），减少 Java 端 BigDecimal 运算。
若需要我协助具体代码重构和批量 SQL 实现，请告知。
Map<String, RecordSnapshot> dbSnapshot = preloadAll();
List<Diff> toUpdate = new ArrayList<>();
List<Map<String,String>> toInsert = new ArrayList<>();
for (JSONObject item : apiData) {
String key = buildKey(...);
RecordSnapshot old = dbSnapshot.get(key);
if (old == null) toInsert.add(map);
else if (hasDiff(old, map)) toUpdate.add(diff);
}
User
ModeUtil.save可以最后完成的时候执行一次，仅仅用于权限重构
Cursor
package com.ljj.scheduleSync;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import weaver.conn.RecordSet;
import dev.soft.lb.util.CommonUtil;
import dev.soft.lb.util.LCUtil;
import dev.soft.lb.util.ModeUtil;
import weaver.general.Util;
import weaver.general.TimeUtil;
import weaver.hrm.company.SubCompanyComInfo;
import weaver.interfaces.schedule.BaseCronJob;
import weaver.general.BaseBean;
import java.text.SimpleDateFormat;
import java.util.HashMap;
import java.util.Map;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Set;
import java.util.HashSet;
import java.util.Arrays;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.UUID;
public class syncCGJSD extends BaseCronJob {
// 防重复执行的标识，避免 10 分钟间隔出现重叠
private static final AtomicBoolean RUNNING = new AtomicBoolean(false);
private static final String COMPANY_ID = "66";
private static final String BILL_TYPE = "12";
private static final String TABLE_NAME = "uf_cgjsd";
private static final int DEFAULT_USER_ID = 1;
private static volatile Integer MODE_ID_CACHE = null;
private static final SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd");
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(
Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
"availableforeigncurrency",
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
)
);
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
@Override
public void execute() {
BaseBean log = new BaseBean();
log.writeLog("采购结算单同步任务开始");
if (!RUNNING.compareAndSet(false, true)) {
log.writeLog("检测到正在执行的采购结算单同步任务，本次调度跳过");
return;
}
try {
RecordSet rs = new RecordSet();
String dwbm = new SubCompanyComInfo().getSubCompanyCode(COMPANY_ID);
String api = LCUtil.getLCBillTypeInfo(BILL_TYPE, rs).get("api_list").toString();
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
JSONObject syncResult = syncDataIncremental(paramJson, rs, api);
if (syncResult.containsKey("updateInfo")) {
log.writeLog(syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("failCount") && syncResult.getInteger("failCount") > 0) {
log.writeLog("失败记录数: " + syncResult.getInteger("failCount"));
}
log.writeLog("采购结算单同步任务完成");
} catch (Exception e) {
log.writeLog("采购结算单同步异常：" + e.getMessage());
} finally {
RUNNING.set(false);
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
private boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true;
}
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue;
}
if (!newData.containsKey(columnName)) {
continue;
}
String newValue = Util.null2String(newData.get(columnName));
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
return true;
}
}
return false;
} catch (Exception e) {
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
private JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
private JSONObject syncDataIncremental(JSONObject param, RecordSet rs, String api) {
JSONObject result = new JSONObject();
int totalCount = 0, newCount = 0, updateCount = 0, skipCount = 0, failCount = 0;
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(param.getString("page") == null ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
if (pageCount == 0) {
break;
}
totalCount += pageCount;
if (datas != null) {
for (int i = 0; i < datas.size(); i++) {
String billnum = "";
String cgjsdid = "";
try {
JSONObject item = datas.getJSONObject(i);
Map<String, String> data = new HashMap<>();
cgjsdid = Util.null2String(item.get("ID"));
billnum = Util.null2String(item.get("BILLNUM"));
// 映射字段
data.put("cgjsdid", cgjsdid);
data.put("billnum", billnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", DATE_FORMAT.format(item.getDate("BILLDATE")));
}
// 金额字段
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
int action = processRecordCompareAndUpsert(data, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
} else if (action == 2) {
newCount++;
} else if (action == -1) {
failCount++;
} else {
skipCount++;
}
} catch (Exception e) {
failCount++;
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
}
}
// 每页处理完成后，执行关联字段更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
// 忽略关联更新异常
}
if (pageCount < Integer.parseInt(pageSize)) {
break;
}
page++;
try {
Thread.sleep(200L);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
break;
}
}
rebuildPermissions(affectedIds);
String syncInfo = "总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount;
result.put("updateInfo", syncInfo);
result.put("updatedDetails", updatedDetails);
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
} catch (Exception e) {
result.put("error", "同步异常: " + e.getMessage());
}
return result;
}
/**
* 先对比所有映射字段，再决定执行更新或插入。
* @return 0=跳过，1=更新，2=新增，-1=失败
*/
private int processRecordCompareAndUpsert(Map<String, String> data,
JSONArray updateDetails,
JSONArray failedRecords,
Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet checkRs = new RecordSet();
checkRs.executeQuery("SELECT * FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (checkRs.next()) {
int existingId = checkRs.getInt("id");
if (hasDataChanged(checkRs, data)) {
JSONArray changes = collectDifferences(checkRs, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
boolean success = applyUpdate(existingId, changes);
if (success) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
return 0;
}
return 0;
}
// 新增记录
Integer newId = insertRecord(data);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + TABLE_NAME + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId()));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
String sql = "INSERT INTO " + TABLE_NAME + "(" + columns + ") VALUES(" + values + ")";
RecordSet insertRs = new RecordSet();
boolean success = insertRs.execute(sql);
if (!success) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
ModeUtil.rebuildModeRights(TABLE_NAME, DEFAULT_USER_ID, affectedIds);
} catch (Exception e) {
new BaseBean().writeLog("采购结算单同步-重构权限失败: " + e.getMessage());
}
}
private int getModeId() {
if (MODE_ID_CACHE != null && MODE_ID_CACHE > 0) {
return MODE_ID_CACHE;
}
synchronized (syncCGJSD.class) {
if (MODE_ID_CACHE == null || MODE_ID_CACHE <= 0) {
MODE_ID_CACHE = Util.getIntValue(CommonUtil.getModeIdByTableName(TABLE_NAME));
}
}
return MODE_ID_CACHE;
}
}
package dev.soft.lb.util;
import weaver.conn.RecordSet;
import weaver.formmode.setup.ModeRightInfo;
import weaver.general.TimeUtil;
import weaver.general.Util;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.UUID;
/**
* 模块工具类
* @author 林斌
* @version V1.0
* @date 2024/11/7 19:21
* 2025-05-08
* 林斌
*/
public class ModeUtil {
//重构所有数据权限
private static ModeRightInfo ModeRightInfo = new ModeRightInfo();
public static boolean saveModeData(String table, int userid, RecordSet rs, Map<String,String> data, List<Map<String,Object>> dtdatas, String uniField){
return saveModeData(table,userid,rs,data,dtdatas,uniField,true);
}
/**
* 保存模块数据
* @param table 主表名
* @param userid 操作用户ID
* @param rs 数据集
* @param data 主表数据
* @param dtdatas 明细行数据（List<Map<String,Object>> Map对象：table明细表，uniField明细表唯一字段,datalist数据集List<Map<String,String>>）
* @param uniField 主表唯一字段
* @return
*/
public static boolean saveModeData(String table, int userid, RecordSet rs, Map<String,String> data, List<Map<String,Object>> dtdatas, String uniField,boolean buildRights){
boolean flag = false;
try {
if(rs==null){
rs = new RecordSet();
}
int modeid = Util.getIntValue(CommonUtil.getModeIdByTableName(table));
String sql = "";
String tpsql = "";
String s1 = "";
String s2 = "";
int billid = 0;
//如果存在唯一字段且唯一字段值不为空
if(data.containsKey(uniField)&&!Util.null2String(data.get(uniField)).equals("")){
sql = "select id from "+table+" where "+uniField+"='"+data.get(uniField)+"'";
rs.execute(sql);
if(rs.next()){
billid = Util.getIntValue(Util.null2String(rs.getString("id")),0);
}
}
if(billid>0){
tpsql = "";
for(String key : data.keySet()){
String value = Util.null2String(data.get(key));
if(value.equals("")){
value = "null";
}else{
value = "'"+value.replace("'","''")+"'";
}
tpsql += tpsql.equals("")?(key+"="+ value):(","+key+"="+value);
}
sql = "update "+table+" set "+tpsql+" where id="+ billid;
rs.execute(sql);
if(buildRights){
//设置共享权限
ModeRightInfo.setNewRight(false);
ModeRightInfo.init();
ModeRightInfo.setModeId(modeid);
ModeRightInfo.rebuildModeDataShareByEdit(userid, modeid, billid);
}
}else{
String nowtime = TimeUtil.getCurrentTimeString();
String createDate = nowtime.substring(0,10);
String createTime = nowtime.substring(11);
String uuid = UUID.randomUUID().toString();
data.put("modeuuid",uuid);
//无数据则执行新增
data.put("formmodeid",modeid+"");
data.put("modedatacreater",userid+"");
data.put("modedatacreatertype","1");
data.put("modedatacreatedate",createDate);
data.put("modedatacreatetime",createTime);
s1 = "";
s2 = "";
for(String key : data.keySet()){
String value = Util.null2String(data.get(key));
if(!value.equals("")){
value = "'"+value.replace("'","''")+"'";
}else{
value = "null";
}
s1 += s1.equals("")?key:","+key;
s2 += s2.equals("")?value:","+value+"";
}
//新增数据
sql = "insert into "+table+"("+s1+") values("+s2+")";
boolean saveflag = rs.execute(sql);
//如果保存成功
if(saveflag) {
//如果需要更新权限，则重构权限
if(buildRights) {
//查询新增的数据
sql = "select id from " + table + " where modeuuid='" + uuid + "'";
rs.execute(sql);
if (rs.next()) {
billid = Util.getIntValue(Util.null2String(rs.getString("id")), 0);
}
//如果新增数据，则重构权限
if (billid > 0) {
//设置共享权限
ModeRightInfo.setNewRight(false);
ModeRightInfo.init();
ModeRightInfo.setModeId(modeid);
ModeRightInfo.rebuildModeDataShareByEdit(userid, modeid, billid);
}
}
}
}
//如果有数据
if(billid>0 && dtdatas!=null && dtdatas.size()>0){
//循环所有明细数据
for(Map<String,Object> item : dtdatas){
//明细表名
String dttable = (String) item.get("table");
//明细唯一字段
String dtuniField = (String) item.get("uniField");
//数据行
List<Map<String,String>> dtlist = (List<Map<String, String>>) item.get("datalist");
if(dtlist!=null && dtlist.size()>0){
for(Map<String,String> subdata : dtlist){
int dtid = 0;
//如果有关键值且关键值不为空
if(subdata.containsKey(dtuniField)&& !Util.null2String(subdata.get(dtuniField)).equals("")){
sql = "select id from "+dttable+" where mainid="+billid+" and "+dtuniField+"='"+subdata.get(dtuniField)+"'";
rs.execute(sql);
if(rs.next()){
dtid = Util.getIntValue(Util.null2String(rs.getString("id")),0);
}
}
//如果明细数据ID大于0，则更新
if(dtid>0){
tpsql = "";
for(String key : subdata.keySet()){
String value = Util.null2String(subdata.get(key));
if(value.equals("")){
value = "null";
}else{
value = "'"+value.replace("'","''")+"'";
}
tpsql += tpsql.equals("")?(key+"="+ value):(","+key+"="+value);
}
sql = "update "+dttable+" set "+tpsql+" where id="+ dtid;
rs.execute(sql);
}else{
subdata.put("mainid",billid+"");
s1 = "";
s2 = "";
for(String key : subdata.keySet()){
String value = Util.null2String(subdata.get(key));
if(!value.equals("")){
value = "'"+value.replace("'","''")+"'";
}else{
value = "null";
}
s1 += s1.equals("")?key:","+key;
s2 += s2.equals("")?value:","+value+"";
}
//新增明细数据
sql = "insert into "+dttable+"("+s1+") values("+s2+")";
rs.execute(sql);
}
}
}
}
}
flag = true;
}catch (Exception e){
throw new RuntimeException("保存模块数据异常："+e.getMessage());
}
return flag;
}
/**
* 批量重构模块数据权限
*/
public static void rebuildModeRights(String table, int userid, Set<Integer> billIds) {
if (billIds == null || billIds.isEmpty()) {
return;
}
int modeid = Util.getIntValue(CommonUtil.getModeIdByTableName(table));
if (modeid <= 0) {
return;
}
ModeRightInfo.setNewRight(false);
ModeRightInfo.init();
ModeRightInfo.setModeId(modeid);
for (Integer billid : billIds) {
if (billid == null || billid <= 0) {
continue;
}
ModeRightInfo.rebuildModeDataShareByEdit(userid, modeid, billid);
}
}
}
package com.ljj.scheduleSync;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import weaver.conn.RecordSet;
import dev.soft.lb.util.CommonUtil;
import dev.soft.lb.util.LCUtil;
import dev.soft.lb.util.ModeUtil;
import weaver.general.Util;
import weaver.general.TimeUtil;
import weaver.formmode.setup.ModeRightInfo;
import weaver.hrm.company.SubCompanyComInfo;
import weaver.interfaces.schedule.BaseCronJob;
import weaver.general.BaseBean;
import java.text.SimpleDateFormat;
import java.util.HashMap;
import java.util.Map;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Set;
import java.util.HashSet;
import java.util.Arrays;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.UUID;
import java.lang.reflect.Method;
public class syncCGJSD extends BaseCronJob {
// 防重复执行的标识，避免 10 分钟间隔出现重叠
private static final AtomicBoolean RUNNING = new AtomicBoolean(false);
private static final String COMPANY_ID = "66";
private static final String BILL_TYPE = "12";
private static final String TABLE_NAME = "uf_cgjsd";
private static final int DEFAULT_USER_ID = 1;
private static volatile Integer MODE_ID_CACHE = null;
private static final SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd");
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(
Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
"availableforeigncurrency",
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
)
);
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
@Override
public void execute() {
BaseBean log = new BaseBean();
log.writeLog("采购结算单同步任务开始");
if (!RUNNING.compareAndSet(false, true)) {
log.writeLog("检测到正在执行的采购结算单同步任务，本次调度跳过");
return;
}
try {
RecordSet rs = new RecordSet();
String dwbm = new SubCompanyComInfo().getSubCompanyCode(COMPANY_ID);
String api = LCUtil.getLCBillTypeInfo(BILL_TYPE, rs).get("api_list").toString();
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
JSONObject syncResult = syncDataIncremental(paramJson, rs, api);
if (syncResult.containsKey("updateInfo")) {
log.writeLog(syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("failCount") && syncResult.getInteger("failCount") > 0) {
log.writeLog("失败记录数: " + syncResult.getInteger("failCount"));
}
log.writeLog("采购结算单同步任务完成");
} catch (Exception e) {
log.writeLog("采购结算单同步异常：" + e.getMessage());
} finally {
RUNNING.set(false);
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
private boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true;
}
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue;
}
if (!newData.containsKey(columnName)) {
continue;
}
String newValue = Util.null2String(newData.get(columnName));
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
return true;
}
}
return false;
} catch (Exception e) {
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
private JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
private JSONObject syncDataIncremental(JSONObject param, RecordSet rs, String api) {
JSONObject result = new JSONObject();
int totalCount = 0, newCount = 0, updateCount = 0, skipCount = 0, failCount = 0;
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(param.getString("page") == null ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
if (pageCount == 0) {
break;
}
totalCount += pageCount;
if (datas != null) {
for (int i = 0; i < datas.size(); i++) {
String billnum = "";
String cgjsdid = "";
try {
JSONObject item = datas.getJSONObject(i);
Map<String, String> data = new HashMap<>();
cgjsdid = Util.null2String(item.get("ID"));
billnum = Util.null2String(item.get("BILLNUM"));
// 映射字段
data.put("cgjsdid", cgjsdid);
data.put("billnum", billnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", DATE_FORMAT.format(item.getDate("BILLDATE")));
}
// 金额字段
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
int action = processRecordCompareAndUpsert(data, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
} else if (action == 2) {
newCount++;
} else if (action == -1) {
failCount++;
} else {
skipCount++;
}
} catch (Exception e) {
failCount++;
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
}
}
// 每页处理完成后，执行关联字段更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
// 忽略关联更新异常
}
if (pageCount < Integer.parseInt(pageSize)) {
break;
}
page++;
try {
Thread.sleep(200L);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
break;
}
}
rebuildPermissions(affectedIds);
String syncInfo = "总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount;
result.put("updateInfo", syncInfo);
result.put("updatedDetails", updatedDetails);
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
} catch (Exception e) {
result.put("error", "同步异常: " + e.getMessage());
}
return result;
}
/**
* 先对比所有映射字段，再决定执行更新或插入。
* @return 0=跳过，1=更新，2=新增，-1=失败
*/
private int processRecordCompareAndUpsert(Map<String, String> data,
JSONArray updateDetails,
JSONArray failedRecords,
Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet checkRs = new RecordSet();
checkRs.executeQuery("SELECT * FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (checkRs.next()) {
int existingId = checkRs.getInt("id");
if (hasDataChanged(checkRs, data)) {
JSONArray changes = collectDifferences(checkRs, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
boolean success = applyUpdate(existingId, changes);
if (success) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
return 0;
}
return 0;
}
// 新增记录
Integer newId = insertRecord(data);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + TABLE_NAME + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId()));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
String sql = "INSERT INTO " + TABLE_NAME + "(" + columns + ") VALUES(" + values + ")";
RecordSet insertRs = new RecordSet();
boolean success = insertRs.execute(sql);
if (!success) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
Method method = ModeUtil.class.getMethod("rebuildModeRights", String.class, int.class, Set.class);
method.invoke(null, TABLE_NAME, DEFAULT_USER_ID, affectedIds);
} catch (NoSuchMethodException e) {
rebuildPermissionsFallback(affectedIds);
} catch (Exception e) {
new BaseBean().writeLog("采购结算单同步-重构权限失败: " + e.getMessage());
}
}
private void rebuildPermissionsFallback(Set<Integer> affectedIds) {
try {
int modeId = getModeId();
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
new BaseBean().writeLog("采购结算单同步-重构权限Fallback失败: " + ex.getMessage());
}
}
private int getModeId() {
if (MODE_ID_CACHE != null && MODE_ID_CACHE > 0) {
return MODE_ID_CACHE;
}
synchronized (syncCGJSD.class) {
if (MODE_ID_CACHE == null || MODE_ID_CACHE <= 0) {
MODE_ID_CACHE = Util.getIntValue(CommonUtil.getModeIdByTableName(TABLE_NAME));
}
}
return MODE_ID_CACHE;
}
}
更新摘要
- 在
syncCGJSD中改为记录所有变更的affectedIds，循环结束后一次性调用rebuildPermissions，避免在 10 万条处理过程中频繁触发ModeUtil.saveModeData，并显式打断分页条件与节流，减少单次任务的 DB/API 压力。
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
...
rebuildPermissions(affectedIds);
String syncInfo = "总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount;
- 将原先逐条
ModeUtil.saveModeData操作替换为自定义applyUpdate/insertRecordSQL，批量写库后仅收集需要重构权限的主键，再在rebuildPermissions中通过反射尝试一次调用新版ModeUtil.rebuildModeRights，若缺失则回退到ModeRightInfo逐条重构，满足“ModeUtil 只在末尾用于权限”的诉求。
boolean success = applyUpdate(existingId, changes);
...
Integer newId = insertRecord(data);
...
private void rebuildPermissions(Set<Integer> affectedIds) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
Method method = ModeUtil.class.getMethod("rebuildModeRights", String.class, int.class, Set.class);
method.invoke(null, TABLE_NAME, DEFAULT_USER_ID, affectedIds);
} catch (NoSuchMethodException e) {
rebuildPermissionsFallback(affectedIds);
} catch (Exception e) {
new BaseBean().writeLog("采购结算单同步-重构权限失败: " + e.getMessage());
}
}
- 在
ModeUtil中新增rebuildModeRights，支持一次性为多个记录重建模式权限，供同步任务末尾统一调用。
public static void rebuildModeRights(String table, int userid, Set<Integer> billIds) {
if (billIds == null || billIds.isEmpty()) {
return;
}
int modeid = Util.getIntValue(CommonUtil.getModeIdByTableName(table));
if (modeid <= 0) {
return;
}
ModeRightInfo.setNewRight(false);
ModeRightInfo.init();
ModeRightInfo.setModeId(modeid);
for (Integer billid : billIds) {
if (billid == null || billid <= 0) {
continue;
}
ModeRightInfo.rebuildModeDataShareByEdit(userid, modeid, billid);
}
}
测试
- 尚未运行自动化或集成测试；请在部署前使用 10 万条接近真实体量的数据手动跑一次任务，确认耗时与权限结果。
后续建议
- 若仍难在 10 s 内完成，可继续引入临时表批量
INSERT … SELECT或 Redis 快照进一步降低 DB 往返。 - 任务上线前建议重点验证：① 新增/更新是否写入
formmodeid等元数据；② 权限重构是否覆盖所有affectedIds。
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
...
rebuildPermissions(affectedIds);
String syncInfo = "总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount;
boolean success = applyUpdate(existingId, changes);
...
Integer newId = insertRecord(data);
...
private void rebuildPermissions(Set<Integer> affectedIds) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
Method method = ModeUtil.class.getMethod("rebuildModeRights", String.class, int.class, Set.class);
method.invoke(null, TABLE_NAME, DEFAULT_USER_ID, affectedIds);
} catch (NoSuchMethodException e) {
rebuildPermissionsFallback(affectedIds);
} catch (Exception e) {
new BaseBean().writeLog("采购结算单同步-重构权限失败: " + e.getMessage());
}
}
public static void rebuildModeRights(String table, int userid, Set<Integer> billIds) {
if (billIds == null || billIds.isEmpty()) {
return;
}
int modeid = Util.getIntValue(CommonUtil.getModeIdByTableName(table));
if (modeid <= 0) {
return;
}
ModeRightInfo.setNewRight(false);
ModeRightInfo.init();
ModeRightInfo.setModeId(modeid);
for (Integer billid : billIds) {
if (billid == null || billid <= 0) {
continue;
}
ModeRightInfo.rebuildModeDataShareByEdit(userid, modeid, billid);
}
}
User
重构不要写在@ModeUtil.java 里面去，就写在同步里面
Cursor
package dev.soft.lb.util;
import weaver.conn.RecordSet;
import weaver.formmode.setup.ModeRightInfo;
import weaver.general.TimeUtil;
import weaver.general.Util;
import java.util.List;
import java.util.Map;
import java.util.UUID;
/**
* 模块工具类
* @author 林斌
* @version V1.0
* @date 2024/11/7 19:21
* 2025-05-08
* 林斌
*/
public class ModeUtil {
//重构所有数据权限
private static ModeRightInfo ModeRightInfo = new ModeRightInfo();
public static boolean saveModeData(String table, int userid, RecordSet rs, Map<String,String> data, List<Map<String,Object>> dtdatas, String uniField){
return saveModeData(table,userid,rs,data,dtdatas,uniField,true);
}
/**
* 保存模块数据
* @param table 主表名
* @param userid 操作用户ID
* @param rs 数据集
* @param data 主表数据
* @param dtdatas 明细行数据（List<Map<String,Object>> Map对象：table明细表，uniField明细表唯一字段,datalist数据集List<Map<String,String>>）
* @param uniField 主表唯一字段
* @return
*/
public static boolean saveModeData(String table, int userid, RecordSet rs, Map<String,String> data, List<Map<String,Object>> dtdatas, String uniField,boolean buildRights){
boolean flag = false;
try {
if(rs==null){
rs = new RecordSet();
}
int modeid = Util.getIntValue(CommonUtil.getModeIdByTableName(table));
String sql = "";
String tpsql = "";
String s1 = "";
String s2 = "";
int billid = 0;
//如果存在唯一字段且唯一字段值不为空
if(data.containsKey(uniField)&&!Util.null2String(data.get(uniField)).equals("")){
sql = "select id from "+table+" where "+uniField+"='"+data.get(uniField)+"'";
rs.execute(sql);
if(rs.next()){
billid = Util.getIntValue(Util.null2String(rs.getString("id")),0);
}
}
if(billid>0){
tpsql = "";
for(String key : data.keySet()){
String value = Util.null2String(data.get(key));
if(value.equals("")){
value = "null";
}else{
value = "'"+value.replace("'","''")+"'";
}
tpsql += tpsql.equals("")?(key+"="+ value):(","+key+"="+value);
}
sql = "update "+table+" set "+tpsql+" where id="+ billid;
rs.execute(sql);
if(buildRights){
//设置共享权限
ModeRightInfo.setNewRight(false);
ModeRightInfo.init();
ModeRightInfo.setModeId(modeid);
ModeRightInfo.rebuildModeDataShareByEdit(userid, modeid, billid);
}
}else{
String nowtime = TimeUtil.getCurrentTimeString();
String createDate = nowtime.substring(0,10);
String createTime = nowtime.substring(11);
String uuid = UUID.randomUUID().toString();
data.put("modeuuid",uuid);
//无数据则执行新增
data.put("formmodeid",modeid+"");
data.put("modedatacreater",userid+"");
data.put("modedatacreatertype","1");
data.put("modedatacreatedate",createDate);
data.put("modedatacreatetime",createTime);
s1 = "";
s2 = "";
for(String key : data.keySet()){
String value = Util.null2String(data.get(key));
if(!value.equals("")){
value = "'"+value.replace("'","''")+"'";
}else{
value = "null";
}
s1 += s1.equals("")?key:","+key;
s2 += s2.equals("")?value:","+value+"";
}
//新增数据
sql = "insert into "+table+"("+s1+") values("+s2+")";
boolean saveflag = rs.execute(sql);
//如果保存成功
if(saveflag) {
//如果需要更新权限，则重构权限
if(buildRights) {
//查询新增的数据
sql = "select id from " + table + " where modeuuid='" + uuid + "'";
rs.execute(sql);
if (rs.next()) {
billid = Util.getIntValue(Util.null2String(rs.getString("id")), 0);
}
//如果新增数据，则重构权限
if (billid > 0) {
//设置共享权限
ModeRightInfo.setNewRight(false);
ModeRightInfo.init();
ModeRightInfo.setModeId(modeid);
ModeRightInfo.rebuildModeDataShareByEdit(userid, modeid, billid);
}
}
}
}
//如果有数据
if(billid>0 && dtdatas!=null && dtdatas.size()>0){
//循环所有明细数据
for(Map<String,Object> item : dtdatas){
//明细表名
String dttable = (String) item.get("table");
//明细唯一字段
String dtuniField = (String) item.get("uniField");
//数据行
List<Map<String,String>> dtlist = (List<Map<String, String>>) item.get("datalist");
if(dtlist!=null && dtlist.size()>0){
for(Map<String,String> subdata : dtlist){
int dtid = 0;
//如果有关键值且关键值不为空
if(subdata.containsKey(dtuniField)&& !Util.null2String(subdata.get(dtuniField)).equals("")){
sql = "select id from "+dttable+" where mainid="+billid+" and "+dtuniField+"='"+subdata.get(dtuniField)+"'";
rs.execute(sql);
if(rs.next()){
dtid = Util.getIntValue(Util.null2String(rs.getString("id")),0);
}
}
//如果明细数据ID大于0，则更新
if(dtid>0){
tpsql = "";
for(String key : subdata.keySet()){
String value = Util.null2String(subdata.get(key));
if(value.equals("")){
value = "null";
}else{
value = "'"+value.replace("'","''")+"'";
}
tpsql += tpsql.equals("")?(key+"="+ value):(","+key+"="+value);
}
sql = "update "+dttable+" set "+tpsql+" where id="+ dtid;
rs.execute(sql);
}else{
subdata.put("mainid",billid+"");
s1 = "";
s2 = "";
for(String key : subdata.keySet()){
String value = Util.null2String(subdata.get(key));
if(!value.equals("")){
value = "'"+value.replace("'","''")+"'";
}else{
value = "null";
}
s1 += s1.equals("")?key:","+key;
s2 += s2.equals("")?value:","+value+"";
}
//新增明细数据
sql = "insert into "+dttable+"("+s1+") values("+s2+")";
rs.execute(sql);
}
}
}
}
}
flag = true;
}catch (Exception e){
throw new RuntimeException("保存模块数据异常："+e.getMessage());
}
return flag;
}
}
package com.ljj.scheduleSync;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import weaver.conn.RecordSet;
import dev.soft.lb.util.CommonUtil;
import dev.soft.lb.util.LCUtil;
import dev.soft.lb.util.ModeUtil;
import weaver.general.Util;
import weaver.general.TimeUtil;
import weaver.formmode.setup.ModeRightInfo;
import weaver.hrm.company.SubCompanyComInfo;
import weaver.interfaces.schedule.BaseCronJob;
import weaver.general.BaseBean;
import java.text.SimpleDateFormat;
import java.util.HashMap;
import java.util.Map;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Set;
import java.util.HashSet;
import java.util.Arrays;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.UUID;
public class syncCGJSD extends BaseCronJob {
// 防重复执行的标识，避免 10 分钟间隔出现重叠
private static final AtomicBoolean RUNNING = new AtomicBoolean(false);
private static final String COMPANY_ID = "66";
private static final String BILL_TYPE = "12";
private static final String TABLE_NAME = "uf_cgjsd";
private static final int DEFAULT_USER_ID = 1;
private static volatile Integer MODE_ID_CACHE = null;
private static final SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd");
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(
Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
"availableforeigncurrency",
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
)
);
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
@Override
public void execute() {
BaseBean log = new BaseBean();
log.writeLog("采购结算单同步任务开始");
if (!RUNNING.compareAndSet(false, true)) {
log.writeLog("检测到正在执行的采购结算单同步任务，本次调度跳过");
return;
}
try {
RecordSet rs = new RecordSet();
String dwbm = new SubCompanyComInfo().getSubCompanyCode(COMPANY_ID);
String api = LCUtil.getLCBillTypeInfo(BILL_TYPE, rs).get("api_list").toString();
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
JSONObject syncResult = syncDataIncremental(paramJson, rs, api);
if (syncResult.containsKey("updateInfo")) {
log.writeLog(syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("failCount") && syncResult.getInteger("failCount") > 0) {
log.writeLog("失败记录数: " + syncResult.getInteger("failCount"));
}
log.writeLog("采购结算单同步任务完成");
} catch (Exception e) {
log.writeLog("采购结算单同步异常：" + e.getMessage());
} finally {
RUNNING.set(false);
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
private boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true;
}
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue;
}
if (!newData.containsKey(columnName)) {
continue;
}
String newValue = Util.null2String(newData.get(columnName));
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
return true;
}
}
return false;
} catch (Exception e) {
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
private JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
private JSONObject syncDataIncremental(JSONObject param, RecordSet rs, String api) {
JSONObject result = new JSONObject();
int totalCount = 0, newCount = 0, updateCount = 0, skipCount = 0, failCount = 0;
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(param.getString("page") == null ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
if (pageCount == 0) {
break;
}
totalCount += pageCount;
if (datas != null) {
for (int i = 0; i < datas.size(); i++) {
String billnum = "";
String cgjsdid = "";
try {
JSONObject item = datas.getJSONObject(i);
Map<String, String> data = new HashMap<>();
cgjsdid = Util.null2String(item.get("ID"));
billnum = Util.null2String(item.get("BILLNUM"));
// 映射字段
data.put("cgjsdid", cgjsdid);
data.put("billnum", billnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", DATE_FORMAT.format(item.getDate("BILLDATE")));
}
// 金额字段
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
int action = processRecordCompareAndUpsert(data, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
} else if (action == 2) {
newCount++;
} else if (action == -1) {
failCount++;
} else {
skipCount++;
}
} catch (Exception e) {
failCount++;
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
}
}
// 每页处理完成后，执行关联字段更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
// 忽略关联更新异常
}
if (pageCount < Integer.parseInt(pageSize)) {
break;
}
page++;
try {
Thread.sleep(200L);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
break;
}
}
rebuildPermissions(affectedIds);
String syncInfo = "总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount;
result.put("updateInfo", syncInfo);
result.put("updatedDetails", updatedDetails);
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
} catch (Exception e) {
result.put("error", "同步异常: " + e.getMessage());
}
return result;
}
/**
* 先对比所有映射字段，再决定执行更新或插入。
* @return 0=跳过，1=更新，2=新增，-1=失败
*/
private int processRecordCompareAndUpsert(Map<String, String> data,
JSONArray updateDetails,
JSONArray failedRecords,
Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet checkRs = new RecordSet();
checkRs.executeQuery("SELECT * FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (checkRs.next()) {
int existingId = checkRs.getInt("id");
if (hasDataChanged(checkRs, data)) {
JSONArray changes = collectDifferences(checkRs, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
boolean success = applyUpdate(existingId, changes);
if (success) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
return 0;
}
return 0;
}
// 新增记录
Integer newId = insertRecord(data);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + TABLE_NAME + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId()));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
String sql = "INSERT INTO " + TABLE_NAME + "(" + columns + ") VALUES(" + values + ")";
RecordSet insertRs = new RecordSet();
boolean success = insertRs.execute(sql);
if (!success) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId();
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
new BaseBean().writeLog("采购结算单同步-重构权限失败: " + ex.getMessage());
}
}
private int getModeId() {
if (MODE_ID_CACHE != null && MODE_ID_CACHE > 0) {
return MODE_ID_CACHE;
}
synchronized (syncCGJSD.class) {
if (MODE_ID_CACHE == null || MODE_ID_CACHE <= 0) {
MODE_ID_CACHE = Util.getIntValue(CommonUtil.getModeIdByTableName(TABLE_NAME));
}
}
return MODE_ID_CACHE;
}
}
package com.ljj.scheduleSync;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import weaver.conn.RecordSet;
import dev.soft.lb.util.CommonUtil;
import dev.soft.lb.util.LCUtil;
import weaver.general.Util;
import weaver.general.TimeUtil;
import weaver.formmode.setup.ModeRightInfo;
import weaver.hrm.company.SubCompanyComInfo;
import weaver.interfaces.schedule.BaseCronJob;
import weaver.general.BaseBean;
import java.text.SimpleDateFormat;
import java.util.HashMap;
import java.util.Map;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.util.Set;
import java.util.HashSet;
import java.util.Arrays;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.UUID;
public class syncCGJSD extends BaseCronJob {
// 防重复执行的标识，避免 10 分钟间隔出现重叠
private static final AtomicBoolean RUNNING = new AtomicBoolean(false);
private static final String COMPANY_ID = "66";
private static final String BILL_TYPE = "12";
private static final String TABLE_NAME = "uf_cgjsd";
private static final int DEFAULT_USER_ID = 1;
private static volatile Integer MODE_ID_CACHE = null;
private static final SimpleDateFormat DATE_FORMAT = new SimpleDateFormat("yyyy-MM-dd");
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(
Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
"availableforeigncurrency",
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
)
);
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
@Override
public void execute() {
BaseBean log = new BaseBean();
log.writeLog("采购结算单同步任务开始");
if (!RUNNING.compareAndSet(false, true)) {
log.writeLog("检测到正在执行的采购结算单同步任务，本次调度跳过");
return;
}
try {
RecordSet rs = new RecordSet();
String dwbm = new SubCompanyComInfo().getSubCompanyCode(COMPANY_ID);
String api = LCUtil.getLCBillTypeInfo(BILL_TYPE, rs).get("api_list").toString();
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
JSONObject syncResult = syncDataIncremental(paramJson, rs, api);
if (syncResult.containsKey("updateInfo")) {
log.writeLog(syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("failCount") && syncResult.getInteger("failCount") > 0) {
log.writeLog("失败记录数: " + syncResult.getInteger("failCount"));
}
log.writeLog("采购结算单同步任务完成");
} catch (Exception e) {
log.writeLog("采购结算单同步异常：" + e.getMessage());
} finally {
RUNNING.set(false);
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
private boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true;
}
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue;
}
if (!newData.containsKey(columnName)) {
continue;
}
String newValue = Util.null2String(newData.get(columnName));
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
return true;
}
}
return false;
} catch (Exception e) {
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
private JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
private JSONObject syncDataIncremental(JSONObject param, RecordSet rs, String api) {
JSONObject result = new JSONObject();
int totalCount = 0, newCount = 0, updateCount = 0, skipCount = 0, failCount = 0;
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(param.getString("page") == null ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
if (pageCount == 0) {
break;
}
totalCount += pageCount;
if (datas != null) {
for (int i = 0; i < datas.size(); i++) {
String billnum = "";
String cgjsdid = "";
try {
JSONObject item = datas.getJSONObject(i);
Map<String, String> data = new HashMap<>();
cgjsdid = Util.null2String(item.get("ID"));
billnum = Util.null2String(item.get("BILLNUM"));
// 映射字段
data.put("cgjsdid", cgjsdid);
data.put("billnum", billnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", DATE_FORMAT.format(item.getDate("BILLDATE")));
}
// 金额字段
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
int action = processRecordCompareAndUpsert(data, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
} else if (action == 2) {
newCount++;
} else if (action == -1) {
failCount++;
} else {
skipCount++;
}
} catch (Exception e) {
failCount++;
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
}
}
// 每页处理完成后，执行关联字段更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
// 忽略关联更新异常
}
if (pageCount < Integer.parseInt(pageSize)) {
break;
}
page++;
try {
Thread.sleep(200L);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
break;
}
}
rebuildPermissions(affectedIds);
String syncInfo = "总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount;
result.put("updateInfo", syncInfo);
result.put("updatedDetails", updatedDetails);
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
} catch (Exception e) {
result.put("error", "同步异常: " + e.getMessage());
}
return result;
}
/**
* 先对比所有映射字段，再决定执行更新或插入。
* @return 0=跳过，1=更新，2=新增，-1=失败
*/
private int processRecordCompareAndUpsert(Map<String, String> data,
JSONArray updateDetails,
JSONArray failedRecords,
Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet checkRs = new RecordSet();
checkRs.executeQuery("SELECT * FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (checkRs.next()) {
int existingId = checkRs.getInt("id");
if (hasDataChanged(checkRs, data)) {
JSONArray changes = collectDifferences(checkRs, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
boolean success = applyUpdate(existingId, changes);
if (success) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
return 0;
}
return 0;
}
// 新增记录
Integer newId = insertRecord(data);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + TABLE_NAME + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId()));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
String sql = "INSERT INTO " + TABLE_NAME + "(" + columns + ") VALUES(" + values + ")";
RecordSet insertRs = new RecordSet();
boolean success = insertRs.execute(sql);
if (!success) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + TABLE_NAME + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId();
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
new BaseBean().writeLog("采购结算单同步-重构权限失败: " + ex.getMessage());
}
}
private int getModeId() {
if (MODE_ID_CACHE != null && MODE_ID_CACHE > 0) {
return MODE_ID_CACHE;
}
synchronized (syncCGJSD.class) {
if (MODE_ID_CACHE == null || MODE_ID_CACHE <= 0) {
MODE_ID_CACHE = Util.getIntValue(CommonUtil.getModeIdByTableName(TABLE_NAME));
}
}
return MODE_ID_CACHE;
}
}
- 已按要求把权限重构逻辑完全放回
syncCGJSD，直接使用ModeRightInfo在同步末尾统一重建，不再依赖ModeUtil扩展方法；相关导入也清理完毕。 ModeUtil恢复原貌，仅保留现有saveModeData能力；同步流程仍通过批量 SQL 更新/插入并在rebuildPermissions中一次性处理权限。- 目前仅剩早已存在的
ModeUtil泛型未检查 warning（旧代码遗留），其余 lint 通过。
如需进一步优化 ModeUtil 的泛型告警或运行端到端测试，请告知。
User
@syncLCCGJSD.jsp 这个jsp保持和@syncCGJSD.java 功能逻辑一致，我去用jsp做测试
Cursor
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename) {
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", data.get("cgjsdid"), data.get("billnum"));
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
// 仅更新差异字段
JSONArray changes = collectDifferences(check, data);
if (changes != null && changes.size() > 0) {
Map<String, String> updateMap = new HashMap<>();
updateMap.put("id", String.valueOf(existingId));
for (int i = 0; i < changes.size(); i++) {
JSONObject c = changes.getJSONObject(i);
String field = Util.null2String(c.getString("field"));
String newValue = Util.null2String(c.getString("newValue"));
updateMap.put(field, newValue);
}
ModeUtil.saveModeData(tablename, 1, rs, updateMap, null, "id", true);
}
System.out.println("[" + tablename + "] 已更新(仅差异字段)，id=" + existingId + ", cgjsdid=" + data.get("cgjsdid") + ", billnum=" + data.get("billnum"));
return 1;
}
System.out.println("[" + tablename + "] 无变化，跳过，id=" + existingId + ", cgjsdid=" + data.get("cgjsdid") + ", billnum=" + data.get("billnum"));
return 0;
}
ModeUtil.saveModeData(tablename, 1, rs, data, null, "cgjsdid", true);
System.out.println("[" + tablename + "] 已新增，cgjsdid=" + data.get("cgjsdid") + ", billnum=" + data.get("billnum"));
return 2;
} catch (Exception e) {
try {
ModeUtil.saveModeData(tablename, 1, rs, data, null, "cgjsdid", true);
System.out.println("[" + tablename + "] 异常后重试已保存(按cgjsdid插入/更新)，cgjsdid=" + data.get("cgjsdid") + ", billnum=" + data.get("billnum") + ", err=" + e.getMessage());
return 1;
} catch (Exception ignore) {
System.out.println("[" + tablename + "] 保存失败并跳过，cgjsdid=" + data.get("cgjsdid") + ", billnum=" + data.get("billnum") + ", err=" + e.getMessage());
return 0;
}
}
}
// 重载：在更新时将差异字段加入 updateDetails 收集器
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename, JSONArray updateDetails) {
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", data.get("cgjsdid"), data.get("billnum"));
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
// 收集字段差异
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", data.get("billnum"));
detail.put("cgjsdid", data.get("cgjsdid"));
detail.put("changes", changes);
updateDetails.add(detail);
}
// 仅更新差异字段（使用系统管理员权限重构）
if (changes != null && changes.size() > 0) {
Map<String, String> updateMap = new HashMap<>();
updateMap.put("id", String.valueOf(existingId));
for (int i = 0; i < changes.size(); i++) {
JSONObject c = changes.getJSONObject(i);
String field = Util.null2String(c.getString("field"));
String newValue = Util.null2String(c.getString("newValue"));
updateMap.put(field, newValue);
}
ModeUtil.saveModeData(tablename, 1, rs, updateMap, null, "id", true);
}
System.out.println("[" + tablename + "] 已更新(仅差异字段)，id=" + existingId + ", billnum=" + data.get("billnum"));
return 1;
}
System.out.println("[" + tablename + "] 无变化，跳过，id=" + existingId + ", billnum=" + data.get("billnum"));
return 0;
}
ModeUtil.saveModeData(tablename, 1, rs, data, null, "cgjsdid", true);
System.out.println("[" + tablename + "] 已新增，billnum=" + data.get("billnum"));
return 2;
} catch (Exception e) {
try {
ModeUtil.saveModeData(tablename, 1, rs, data, null, "cgjsdid", true);
System.out.println("[" + tablename + "] 异常后重试已保存(按cgjsdid插入/更新)，billnum=" + data.get("billnum") + ", err=" + e.getMessage());
return 1;
} catch (Exception ignore) {
System.out.println("[" + tablename + "] 保存失败并跳过，billnum=" + data.get("billnum") + ", err=" + e.getMessage());
return 0;
}
}
}
// 重载：同时支持 updateDetails 和 failedRecords 收集器，并添加调试日志
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename, JSONArray updateDetails, JSONArray failedRecords) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
// 创建调试日志对象
JSONArray debugLogs = new JSONArray();
System.out.println("[权限重构] 使用系统管理员(userid=1)进行权限重构");
try {
debugLogs.add("[开始] billnum=" + billnum + ", cgjsdid=" + cgjsdid);
System.out.println("[数据检查开始] billnum=" + billnum + ", cgjsdid=" + cgjsdid);
RecordSet check = new RecordSet();
String checkSql = "SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?";
System.out.println("[执行查询] SQL: " + checkSql + ", 参数: cgjsdid=" + cgjsdid + ", billnum=" + billnum);
check.executeQuery(checkSql, cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
System.out.println("[数据已存在] id=" + existingId + ", billnum=" + billnum + ", 开始比较字段...");
if (hasDataChanged(check, data)) {
System.out.println("[检测到变化] billnum=" + billnum + ", 收集差异字段...");
// 收集字段差异
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
// 仅更新差异字段
if (changes != null && changes.size() > 0) {
System.out.println("[准备更新] billnum=" + billnum + ", 差异字段数=" + changes.size());
Map<String, String> updateMap = new HashMap<>();
updateMap.put("id", String.valueOf(existingId));
for (int i = 0; i < changes.size(); i++) {
JSONObject c = changes.getJSONObject(i);
String field = Util.null2String(c.getString("field"));
String newValue = Util.null2String(c.getString("newValue"));
updateMap.put(field, newValue);
System.out.println("[更新字段] " + field + ": " + c.getString("oldValue") + " -> " + newValue);
}
try {
// 使用 ModeUtil.saveModeData 更新并自动重构权限
RecordSet updateRs = new RecordSet();
boolean updateResult = ModeUtil.saveModeData(tablename, 1, updateRs, updateMap, null, "id", true);
System.out.println("[更新成功] billnum=" + billnum + ", id=" + existingId + ", 更新返回值=" + updateResult + " (已自动重构权限)");
} catch (Exception saveEx) {
System.out.println("[!!!更新失败!!!] billnum=" + billnum + ", 错误: " + saveEx.getMessage());
saveEx.printStackTrace();
// 记录失败详情
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("operation", "update");
failedRecord.put("errorMessage", saveEx.getMessage());
failedRecord.put("errorType", saveEx.getClass().getName());
failedRecords.add(failedRecord);
}
return -1;
}
}
System.out.println("[" + tablename + "] 已更新(仅差异字段)，id=" + existingId + ", billnum=" + billnum);
return 1;
}
System.out.println("[" + tablename + "] 无变化，跳过，id=" + existingId + ", billnum=" + billnum);
return 0;
}
// 数据不存在，执行新增
System.out.println("[数据不存在] billnum=" + billnum + ", 准备新增...");
System.out.println("[新增数据详情] cgjsdid=" + cgjsdid + ", billnum=" + billnum);
System.out.println("[关键字段] WLDW=" + data.get("WLDW") + ", DEPARTMENTID=" + data.get("DEPARTMENTID") + ", amount=" + data.get("amount"));
// 打印所有要保存的字段
System.out.println("[准备插入的完整数据] 共 " + data.size() + " 个字段:");
for (Map.Entry<String, String> entry : data.entrySet()) {
System.out.println(" " + entry.getKey() + " = " + entry.getValue());
}
try {
System.out.println("[调用 ModeUtil.saveModeData] 表名=" + tablename + ", modeid=1, 主键字段=cgjsdid");
// 尝试获取保存前的记录数
RecordSet countBefore = new RecordSet();
countBefore.executeQuery("SELECT COUNT(*) as cnt FROM " + tablename);
int countBeforeInsert = 0;
if (countBefore.next()) {
countBeforeInsert = countBefore.getInt("cnt");
}
System.out.println("[保存前记录数] " + countBeforeInsert);
// 关键检查：看数据库中是否已存在该 cgjsdid
RecordSet checkCgjsdid = new RecordSet();
checkCgjsdid.executeQuery("SELECT id, billnum FROM " + tablename + " WHERE cgjsdid = ?", cgjsdid);
if (checkCgjsdid.next()) {
int existingId = checkCgjsdid.getInt("id");
String existingBillnum = checkCgjsdid.getString("billnum");
System.out.println("[!!!警告!!!] 数据库中已存在 cgjsdid=" + cgjsdid + ", id=" + existingId + ", billnum=" + existingBillnum);
System.out.println("[!!!冲突!!!] 当前要插入的 billnum=" + billnum + ", 但数据库中该 cgjsdid 对应的 billnum=" + existingBillnum);
if (!billnum.equals(existingBillnum)) {
System.out.println("[!!!严重错误!!!] cgjsdid 相同但 billnum 不同，这是数据异常！");
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("operation", "insert_cgjsdid_conflict");
failedRecord.put("errorMessage", "cgjsdid 在数据库中已存在(id=" + existingId + ", billnum=" + existingBillnum + ")，但 billnum 不匹配");
failedRecord.put("existingId", existingId);
failedRecord.put("existingBillnum", existingBillnum);
failedRecord.put("debugInfo", "数据库中已存在相同cgjsdid但不同billnum的记录");
failedRecords.add(failedRecord);
}
return -1;
} else {
System.out.println("[提示] cgjsdid 和 billnum 都匹配，记录已存在，跳过插入");
return 0;
}
}
System.out.println("[确认] cgjsdid 在数据库中不存在，可以新增");
// 过滤掉不存在的字段，然后使用 ModeUtil.saveModeData 自动处理权限
System.out.println("[准备使用ModeUtil.saveModeData插入并重构权限]");
try {
// 获取表的所有列名（避免插入不存在的字段）
Set<String> tableColumns = new HashSet<>();
RecordSet columnRs = new RecordSet();
columnRs.executeQuery("SHOW COLUMNS FROM " + tablename);
while (columnRs.next()) {
String columnName = columnRs.getString("Field");
tableColumns.add(columnName.toLowerCase());
}
System.out.println("[表字段数量] " + tableColumns.size());
// 创建过滤后的数据（只包含表中存在的字段）
Map<String, String> filteredData = new HashMap<>();
int skippedCount = 0;
for (Map.Entry<String, String> entry : data.entrySet()) {
String fieldName = entry.getKey();
String fieldValue = entry.getValue();
// 检查字段是否存在于表中
if (!tableColumns.contains(fieldName.toLowerCase())) {
System.out.println("[跳过不存在的字段] " + fieldName);
skippedCount++;
continue;
}
filteredData.put(fieldName, fieldValue);
}
System.out.println("[跳过字段总数] " + skippedCount);
System.out.println("[有效字段数量] " + filteredData.size());
// 使用 ModeUtil.saveModeData，自动处理权限重构（buildRights=true）
RecordSet saveRs = new RecordSet();
boolean saveResult = ModeUtil.saveModeData(tablename, 1, saveRs, filteredData, null, "cgjsdid", true);
System.out.println("[ModeUtil.saveModeData 执行结果] " + saveResult);
if (!saveResult) {
System.out.println("[!!!插入失败!!!] ModeUtil.saveModeData 返回 false");
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("operation", "insert_modeutil_failed");
failedRecord.put("errorMessage", "ModeUtil.saveModeData 返回 false，可能是SQL执行失败");
failedRecord.put("filteredFieldCount", filteredData.size());
failedRecords.add(failedRecord);
}
return -1;
}
System.out.println("[插入成功] 使用ModeUtil.saveModeData插入并已自动重构权限，billnum=" + billnum);
} catch (Exception insertEx) {
System.out.println("[!!!插入异常!!!] " + insertEx.getMessage());
insertEx.printStackTrace();
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("operation", "insert_native_sql_exception");
failedRecord.put("errorMessage", "原生SQL插入异常: " + insertEx.getMessage());
failedRecord.put("errorType", insertEx.getClass().getName());
failedRecords.add(failedRecord);
}
return -1;
}
// 检查保存后的记录数
RecordSet countAfter = new RecordSet();
countAfter.executeQuery("SELECT COUNT(*) as cnt FROM " + tablename);
int countAfterInsert = 0;
if (countAfter.next()) {
countAfterInsert = countAfter.getInt("cnt");
}
System.out.println("[保存后记录数] " + countAfterInsert + " (增加了 " + (countAfterInsert - countBeforeInsert) + " 条)");
// 验证是否真的插入成功
RecordSet verify = new RecordSet();
String verifySql = "SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?";
System.out.println("[验证查询SQL] " + verifySql);
System.out.println("[验证查询参数] cgjsdid=" + cgjsdid + ", billnum=" + billnum);
verify.executeQuery(verifySql, cgjsdid, billnum);
if (verify.next()) {
int newId = verify.getInt("id");
System.out.println("[验证成功] 新记录已插入，id=" + newId + ", billnum=" + billnum);
} else {
System.out.println("[!!!验证失败!!!] 新记录未找到，billnum=" + billnum);
// 尝试只用cgjsdid查询
RecordSet verify2 = new RecordSet();
verify2.executeQuery("SELECT id, billnum FROM " + tablename + " WHERE cgjsdid = ?", cgjsdid);
if (verify2.next()) {
System.out.println("[!!!发现问题!!!] 用cgjsdid能找到记录: id=" + verify2.getInt("id") + ", billnum=" + verify2.getString("billnum"));
} else {
System.out.println("[确认] 用cgjsdid也找不到记录");
}
// 尝试只用billnum查询
RecordSet verify3 = new RecordSet();
verify3.executeQuery("SELECT id, cgjsdid FROM " + tablename + " WHERE billnum = ?", billnum);
if (verify3.next()) {
System.out.println("[!!!发现问题!!!] 用billnum能找到记录: id=" + verify3.getInt("id") + ", cgjsdid=" + verify3.getString("cgjsdid"));
} else {
System.out.println("[确认] 用billnum也找不到记录");
}
// 查询最后插入的记录
RecordSet lastInsert = new RecordSet();
lastInsert.executeQuery("SELECT id, cgjsdid, billnum FROM " + tablename + " ORDER BY id DESC LIMIT 1");
if (lastInsert.next()) {
System.out.println("[最后一条记录] id=" + lastInsert.getInt("id") + ", cgjsdid=" + lastInsert.getString("cgjsdid") + ", billnum=" + lastInsert.getString("billnum"));
}
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("operation", "insert_verify_failed");
failedRecord.put("errorMessage", "保存后验证失败，数据库中未找到该记录。记录数变化：" + (countAfterInsert - countBeforeInsert));
failedRecord.put("allFields", data.toString());
failedRecords.add(failedRecord);
}
return -1;
}
} catch (Exception saveEx) {
System.out.println("[!!!新增失败!!!] billnum=" + billnum + ", 错误: " + saveEx.getMessage());
System.out.println("[完整堆栈]");
saveEx.printStackTrace();
// 记录失败详情
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("operation", "insert");
failedRecord.put("errorMessage", saveEx.getMessage());
failedRecord.put("errorType", saveEx.getClass().getName());
failedRecords.add(failedRecord);
}
return -1;
}
System.out.println("[" + tablename + "] 已新增，billnum=" + billnum);
return 2;
} catch (Exception e) {
System.out.println("[!!!处理异常!!!] billnum=" + billnum + ", cgjsdid=" + cgjsdid);
System.out.println("[异常信息] " + e.getMessage());
e.printStackTrace();
try {
System.out.println("[尝试重试保存] billnum=" + billnum);
ModeUtil.saveModeData(tablename, 1, rs, data, null, "cgjsdid", true);
System.out.println("[" + tablename + "] 异常后重试已保存(按cgjsdid插入/更新)，billnum=" + billnum);
return 1;
} catch (Exception retryEx) {
System.out.println("[!!!重试也失败!!!] billnum=" + billnum + ", 错误: " + retryEx.getMessage());
retryEx.printStackTrace();
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("operation", "retry_failed");
failedRecord.put("errorMessage", "首次失败: " + e.getMessage() + "; 重试失败: " + retryEx.getMessage());
failedRecord.put("errorType", retryEx.getClass().getName());
failedRecords.add(failedRecord);
}
System.out.println("[" + tablename + "] 保存失败并跳过，billnum=" + billnum);
return -1;
}
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
private static final int DEFAULT_USER_ID = 1;
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
private static final int DEFAULT_USER_ID = 1;
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
private static final int DEFAULT_USER_ID = 1;
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
rebuildPermissions(affectedIds, tablename);
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
- JSP 同步入口与
syncCGJSD对齐：接口分页时累计affectedIds，每条记录通过新的processRecordCompareAndUpsert先比对再 SQL 插/更新，并在分页结束后一次性调用ModeRightInfo重构权限，避免循环内多次ModeUtil.save。
Set<Integer> affectedIds = new HashSet<>();
...
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
...
rebuildPermissions(affectedIds, tablename);
- 引入与 Java 版一致的工具方法：
applyUpdate/insertRecord直接组装 SQL，构造modeuuid与formmodeid元数据，并提供toSqlValue、getModeId、rebuildPermissions等辅助，完全抛弃ModeUtil.saveModeData的逐条调用。
private static final int DEFAULT_USER_ID = 1;
...
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) { ... }
private Integer insertRecord(Map<String, String> data, String tablename) { ... }
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) { ... }
现在 JSP 调试页面与定时任务共享同一套增量同步与权限重构逻辑，可直接用于联调测试。
Set<Integer> affectedIds = new HashSet<>();
...
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
...
rebuildPermissions(affectedIds, tablename);
private static final int DEFAULT_USER_ID = 1;
...
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) { ... }
private Integer insertRecord(Map<String, String> data, String tablename) { ... }
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) { ... }
User
@syncLCCGJSD.jsp 这里的分页调整为多少可以减少消耗时间（以10万条数据为准，目前2000条数据需要消耗3.4MB内存）
Cursor
- 你现在单页 2 000 条就吃掉 3.4 MB（≈1.7 KB/条）。如果线性估算，10 000 条就要 17 MB，20 000 条约 34 MB，40 000 条超过 68 MB。JSP 容器通常 512 MB～1 GB 堆，分页太大很容易被单页 JSON 打爆或触发 GC pauses。
- 另一方面，接口响应一次要 3‑4 s。如果把
pagesize设得太小，比如 500 条，就得发 200 次请求（100 000 ÷ 500），网络与序列化的累计时间会远超 10 s。
推荐策略：pagesize 设在 3 000～5 000 之间
- 单页内存大约 5 MB～8.5 MB，可控。
- 100 000 条分 20～34 页，请求时长 ≈ 20×4 s = 80 s（最坏）或更少，如果接口允许并发，可以多线程拉取多页进一步压缩时间。
- 当你改完分页后，配合 JSP 版的本地 upsert & 权限批处理，本地处理时间基本就是 SQL 执行＋权限重构，不再受单页记录数的平方级影响。
如果需要更激进的压缩时间，可以：
- 让接口端支持
lastModified增量过滤，常规 10 min 周期只处理少量记录； - 或者将页数据写入临时表，再在数据库里做
INSERT ... SELECT和UPDATE ... JOIN，这样数据分批写入不会占用 JSP 内存。
User
这是200条数据的实际请求时长和占用 410ms 339.42K
Cursor
- 200 条 → 0.34 MB / 410 ms。按线性估算：
- 2 000 条 ≈ 3.4 MB / 4.1 s
- 5 000 条 ≈ 8.5 MB / 10.3 s
- 10 000 条 ≈ 17 MB / 20.5 s
- 结合 JSP 堆内存与接口 3‑4 s RTT，建议测试时把
pagesize设在 2 000～4 000 区间：内存 <7 MB，单页时长 <8 s，还能把 10 万条控制在 25～50 个请求。 - 若要进一步缩短总时长，优先：① 引入增量条件减少总条数；② 允许并行请求多页；③ 把页数据直接写临时表，由数据库批量更新，JSP 只做调度。
User
使用多线程进行，jsp加载的时候自动创建10个线程发起请求，如果线程中请求返回的内容为3003则停止继续发起，并销毁线程。如果自动创建的线程中还没有请求到3003则继续递增发起，最大线程数20，如果超过20则就先处理完这20个线程的内容后销毁了继续发起，直到处理结束为止
Cursor
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%@ page import="java.util.concurrent.Callable" %>
<%@ page import="java.util.concurrent.CompletionService" %>
<%@ page import="java.util.concurrent.ExecutorCompletionService" %>
<%@ page import="java.util.concurrent.ExecutorService" %>
<%@ page import="java.util.concurrent.Executors" %>
<%@ page import="java.util.concurrent.Future" %>
<%@ page import="java.util.concurrent.atomic.AtomicBoolean" %>
<%@ page import="java.util.concurrent.atomic.AtomicInteger" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
private static final int DEFAULT_USER_ID = 1;
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
rebuildPermissions(affectedIds, tablename);
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%@ page import="java.util.concurrent.Callable" %>
<%@ page import="java.util.concurrent.CompletionService" %>
<%@ page import="java.util.concurrent.ExecutorCompletionService" %>
<%@ page import="java.util.concurrent.ExecutorService" %>
<%@ page import="java.util.concurrent.Executors" %>
<%@ page import="java.util.concurrent.Future" %>
<%@ page import="java.util.concurrent.atomic.AtomicBoolean" %>
<%@ page import="java.util.concurrent.atomic.AtomicInteger" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
private static final int DEFAULT_USER_ID = 1;
private static final int INITIAL_THREAD_COUNT = 10;
private static final int MAX_THREAD_COUNT = 20;
private static class PageResult {
final int page;
final JSONObject response;
PageResult(int page, JSONObject response) {
this.page = page;
this.response = response;
}
}
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
rebuildPermissions(affectedIds, tablename);
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%@ page import="java.util.concurrent.Callable" %>
<%@ page import="java.util.concurrent.CompletionService" %>
<%@ page import="java.util.concurrent.ExecutorCompletionService" %>
<%@ page import="java.util.concurrent.ExecutorService" %>
<%@ page import="java.util.concurrent.Executors" %>
<%@ page import="java.util.concurrent.Future" %>
<%@ page import="java.util.concurrent.atomic.AtomicBoolean" %>
<%@ page import="java.util.concurrent.atomic.AtomicInteger" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
private static final int DEFAULT_USER_ID = 1;
private static final int INITIAL_THREAD_COUNT = 10;
private static final int MAX_THREAD_COUNT = 20;
private static class PageResult {
final int page;
final JSONObject response;
PageResult(int page, JSONObject response) {
this.page = page;
this.response = response;
}
}
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
private boolean submitPageTask(CompletionService<PageResult> completionService,
AtomicInteger nextPage,
AtomicBoolean stopFlag,
String api,
String dwbh,
String pageSize) {
if (stopFlag.get()) {
return false;
}
completionService.submit(new Callable<PageResult>() {
@Override
public PageResult call() throws Exception {
if (stopFlag.get()) {
return null;
}
int pageIndex = nextPage.getAndIncrement();
JSONObject req = new JSONObject();
req.put("dwbh", dwbh);
req.put("pagesize", pageSize);
req.put("page", String.valueOf(pageIndex));
JSONObject resp = LCUtil.doPost(api, req);
if (resp != null && resp.containsKey("code") && resp.getIntValue("code") == 3003) {
stopFlag.set(true);
}
return new PageResult(pageIndex, resp);
}
});
return true;
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
// 收集差异更新与新增的单据编号（billnum）
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
// 收集每条更新的字段变更详情
JSONArray updatedDetails = new JSONArray();
// 收集保存失败的记录详情
JSONArray failedRecords = new JSONArray();
// 收集调试日志（仅记录失败记录的详细过程）
JSONArray debugLogs = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
try {
int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
while (true) {
param.put("page", String.valueOf(page));
param.put("pagesize", pageSize);
JSONObject rjson = LCUtil.doPost(api, param);
result = rjson;
// 结束条件
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
break;
}
if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
break;
}
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
// 映射字段
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
System.out.println("========================================");
System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
// 金额字段：统一两位小数
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
// availableforeigncurrency 字段在数据库表中不存在，已跳过
// data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
System.out.println("[保存失败] billnum: " + currentBillnum);
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
System.out.println("[错误信息] " + ee.getMessage());
System.out.println("[堆栈跟踪开始]");
ee.printStackTrace();
System.out.println("[堆栈跟踪结束]");
System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
// 记录失败详情
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
// 每页结束后可执行一次关联更新
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
}
System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
page++;
}
rebuildPermissions(affectedIds, tablename);
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
result.put("updateInfo", syncInfo);
System.out.println(syncInfo);
System.out.println("========================================");
System.out.println("同步完成汇总:");
System.out.println("总数: " + totalCount);
System.out.println("新增: " + newCount);
System.out.println("更新: " + updateCount);
System.out.println("跳过: " + skipCount);
System.out.println("失败: " + failCount);
if (failCount > 0) {
System.out.println("失败记录详情:");
for (int i = 0; i < failedRecords.size(); i++) {
JSONObject failed = failedRecords.getJSONObject(i);
System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
", cgjsdid=" + failed.getString("cgjsdid") +
", operation=" + failed.getString("operation") +
", error=" + failed.getString("errorMessage"));
}
}
System.out.println("========================================");
// 将差异与新增的单据编号（billnum）放入结果
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
// 返回每条更新的具体字段变化
result.put("updatedDetails", updatedDetails);
// 返回失败记录
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
// 返回调试日志
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
*** Begin Patch
*** Update File: /Users/meaili/weaver/ecologydemo/sx/webapp/syncLCCGJSD.jsp
@@
-public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
- JSONObject result = new JSONObject();
- int totalCount = 0;
- int newCount = 0;
- int updateCount = 0;
- int skipCount = 0;
- int failCount = 0;
- // 收集差异更新与新增的单据编号（billnum）
- JSONArray diffBillnums = new JSONArray();
- JSONArray newBillnums = new JSONArray();
- // 收集每条更新的字段变更详情
- JSONArray updatedDetails = new JSONArray();
- // 收集保存失败的记录详情
- JSONArray failedRecords = new JSONArray();
- // 收集调试日志（仅记录失败记录的详细过程）
- JSONArray debugLogs = new JSONArray();
- Set<Integer> affectedIds = new HashSet<>();
- try {
- int page = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
- String pageSize = Util.null2String(param.getString("pagesize"));
- if ("".equals(pageSize)) pageSize = "10000";
-
- while (true) {
- param.put("page", String.valueOf(page));
- param.put("pagesize", pageSize);
-
- JSONObject rjson = LCUtil.doPost(api, param);
- result = rjson;
-
- // 结束条件
- if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
- System.out.println("分页已结束，code=3003，停止请求。最后页码：" + page);
- break;
- }
- if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
- System.out.println("接口返回非成功code：" + rjson.getString("code") + "，页码：" + page + "，停止。");
- break;
- }
-
- JSONArray datas = rjson.getJSONArray("data");
- int pageCount = (datas == null) ? 0 : datas.size();
- totalCount += pageCount;
- System.out.println("第 " + page + " 页获取到 " + pageCount + " 条数据，开始增量同步...");
-
- if (datas != null) {
- for (int x = 0; x < datas.size(); x++) {
- JSONObject item = datas.getJSONObject(x);
- Map<String, String> data = new HashMap<>();
- String currentBillnum = "";
- String currentCgjsdid = "";
- try {
- // 映射字段
- currentCgjsdid = Util.null2String(item.get("ID"));
- currentBillnum = Util.null2String(item.get("BILLNUM"));
-
- System.out.println("========================================");
- System.out.println("[开始处理] 单据编号: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
-
- data.put("cgjsdid", currentCgjsdid);
- data.put("billnum", currentBillnum);
- data.put("WLDW", Util.null2String(item.get("WLDW")));
- data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
- data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
- data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
- data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
- data.put("BMBH", Util.null2String(item.get("BMBH")));
- data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
- data.put("summary", Util.null2String(item.get("SUMMARY")));
- data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
- data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
- data.put("contractid", Util.null2String(item.get("CONTRACTID")));
- data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
- data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
- data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
- data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
- data.put("dwbh", Util.null2String(item.get("DWBH")));
- data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
- data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
- data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
- data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
- data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
- data.put("Kxxz", Util.null2String(item.get("KXXZ")));
- data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
- data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
- data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
- data.put("XMID", Util.null2String(item.get("XMID")));
- data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
- data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
- data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
- data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
- if (item.get("BILLDATE") != null) {
- data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
- }
- data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
- data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
-
- // 金额字段：统一两位小数
- data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
- data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
- data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
- data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
- data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
- data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
- data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
- data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
- // availableforeigncurrency 字段在数据库表中不存在，已跳过
- // data.put("availableforeigncurrency", formatTo2Decimals(item.getString("AVAILABLEFOREIGNCURRENCY")));
-
- System.out.println("[字段映射完成] billnum: " + currentBillnum + ", 准备保存...");
-
- int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
- if (action == 1) {
- updateCount++;
- diffBillnums.add(data.get("billnum"));
- } else if (action == 2) {
- newCount++;
- newBillnums.add(data.get("billnum"));
- } else if (action == -1) {
- failCount++;
- System.out.println("[保存失败] billnum: " + currentBillnum);
- } else {
- skipCount++;
- }
- } catch (Exception ee) {
- failCount++;
- System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
- System.out.println("[严重错误] billnum: " + currentBillnum + ", cgjsdid: " + currentCgjsdid);
- System.out.println("[错误信息] " + ee.getMessage());
- System.out.println("[堆栈跟踪开始]");
- ee.printStackTrace();
- System.out.println("[堆栈跟踪结束]");
- System.out.println("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!");
-
- // 记录失败详情
- JSONObject failedRecord = new JSONObject();
- failedRecord.put("billnum", currentBillnum);
- failedRecord.put("cgjsdid", currentCgjsdid);
- failedRecord.put("errorMessage", ee.getMessage());
- failedRecord.put("errorType", ee.getClass().getName());
- failedRecords.add(failedRecord);
- }
- }
- }
-
- // 每页结束后可执行一次关联更新
- try {
- rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
- rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
- rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
- rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
- rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
- } catch (Exception ex) {
- ex.printStackTrace();
- result.put("updateError", "第" + page + "页 关联字段更新异常：" + ex.getMessage());
- }
-
- System.out.println("第 " + page + " 页处理完成，累计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount);
- page++;
- }
-
- String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 分页处理完成";
- result.put("updateInfo", syncInfo);
- System.out.println(syncInfo);
- System.out.println("========================================");
- System.out.println("同步完成汇总:");
- System.out.println("总数: " + totalCount);
- System.out.println("新增: " + newCount);
- System.out.println("更新: " + updateCount);
- System.out.println("跳过: " + skipCount);
- System.out.println("失败: " + failCount);
- if (failCount > 0) {
- System.out.println("失败记录详情:");
- for (int i = 0; i < failedRecords.size(); i++) {
- JSONObject failed = failedRecords.getJSONObject(i);
- System.out.println(" [" + (i+1) + "] billnum=" + failed.getString("billnum") +
- ", cgjsdid=" + failed.getString("cgjsdid") +
- ", operation=" + failed.getString("operation") +
- ", error=" + failed.getString("errorMessage"));
- }
- }
- System.out.println("========================================");
-
- // 将差异与新增的单据编号（billnum）放入结果
- result.put("diffBillnums", diffBillnums);
- result.put("newBillnums", newBillnums);
- // 返回每条更新的具体字段变化
- result.put("updatedDetails", updatedDetails);
- // 返回失败记录
- if (failCount > 0) {
- result.put("failedRecords", failedRecords);
- result.put("failCount", failCount);
- }
- // 返回调试日志
- if (debugLogs.size() > 0) {
- result.put("debugLogs", debugLogs);
- }
- } catch (Exception e) {
- e.printStackTrace();
- result.put("error", "增量同步数据异常: " + e.getMessage());
- }
- return result;
-}
+public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
+ JSONObject result = new JSONObject();
+ int totalCount = 0;
+ int newCount = 0;
+ int updateCount = 0;
+ int skipCount = 0;
+ int failCount = 0;
+ JSONArray diffBillnums = new JSONArray();
+ JSONArray newBillnums = new JSONArray();
+ JSONArray updatedDetails = new JSONArray();
+ JSONArray failedRecords = new JSONArray();
+ JSONArray debugLogs = new JSONArray();
+ Set<Integer> affectedIds = new HashSet<>();
+
+ String dwbh = Util.null2String(param.getString("dwbh"));
+ String pageSize = Util.null2String(param.getString("pagesize"));
+ if ("".equals(pageSize)) pageSize = "10000";
+ int startPage = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
+
+ AtomicInteger nextPage = new AtomicInteger(startPage);
+ AtomicBoolean stopFlag = new AtomicBoolean(false);
+ ExecutorService executor = Executors.newFixedThreadPool(MAX_THREAD_COUNT);
+ CompletionService<PageResult> completionService = new ExecutorCompletionService<>(executor);
+
+ int currentCap = INITIAL_THREAD_COUNT;
+ int activeTasks = 0;
+ try {
+ for (int i = 0; i < currentCap; i++) {
+ if (submitPageTask(completionService, nextPage, stopFlag, api, dwbh, pageSize)) {
+ activeTasks++;
+ }
+ }
+
+ while (activeTasks > 0) {
+ Future<PageResult> future = completionService.take();
+ PageResult pageResult = null;
+ try {
+ pageResult = future.get();
+ } catch (Exception taskEx) {
+ failCount++;
+ JSONObject failedRecord = new JSONObject();
+ failedRecord.put("billnum", "");
+ failedRecord.put("cgjsdid", "");
+ failedRecord.put("errorMessage", "线程执行异常: " + taskEx.getMessage());
+ failedRecord.put("errorType", taskEx.getClass().getName());
+ failedRecords.add(failedRecord);
+ }
+ activeTasks--;
+
+ if (pageResult != null && pageResult.response != null && !stopFlag.get()) {
+ JSONObject rjson = pageResult.response;
+ if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
+ stopFlag.set(true);
+ } else if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
+ stopFlag.set(true);
+ result.put("updateError", "第" + pageResult.page + "页接口返回异常code: " + rjson.getString("code"));
+ } else {
+ JSONArray datas = rjson.getJSONArray("data");
+ int pageCount = (datas == null) ? 0 : datas.size();
+ totalCount += pageCount;
+ System.out.println("多线程处理 - 第 " + pageResult.page + " 页获取到 " + pageCount + " 条数据");
+
+ if (datas != null) {
+ for (int x = 0; x < datas.size(); x++) {
+ JSONObject item = datas.getJSONObject(x);
+ Map<String, String> data = new HashMap<>();
+ String currentBillnum = "";
+ String currentCgjsdid = "";
+ try {
+ currentCgjsdid = Util.null2String(item.get("ID"));
+ currentBillnum = Util.null2String(item.get("BILLNUM"));
+
+ data.put("cgjsdid", currentCgjsdid);
+ data.put("billnum", currentBillnum);
+ data.put("WLDW", Util.null2String(item.get("WLDW")));
+ data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
+ data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
+ data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
+ data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
+ data.put("BMBH", Util.null2String(item.get("BMBH")));
+ data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
+ data.put("summary", Util.null2String(item.get("SUMMARY")));
+ data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
+ data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
+ data.put("contractid", Util.null2String(item.get("CONTRACTID")));
+ data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
+ data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
+ data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
+ data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
+ data.put("dwbh", Util.null2String(item.get("DWBH")));
+ data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
+ data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
+ data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
+ data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
+ data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
+ data.put("Kxxz", Util.null2String(item.get("KXXZ")));
+ data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
+ data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
+ data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
+ data.put("XMID", Util.null2String(item.get("XMID")));
+ data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
+ data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
+ data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
+ data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
+ if (item.get("BILLDATE") != null) {
+ data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
+ }
+ data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
+ data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
+
+ data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
+ data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
+ data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
+ data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
+ data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
+ data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
+ data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
+ data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
+
+ int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
+ if (action == 1) {
+ updateCount++;
+ diffBillnums.add(data.get("billnum"));
+ } else if (action == 2) {
+ newCount++;
+ newBillnums.add(data.get("billnum"));
+ } else if (action == -1) {
+ failCount++;
+ } else {
+ skipCount++;
+ }
+ } catch (Exception ee) {
+ failCount++;
+ JSONObject failedRecord = new JSONObject();
+ failedRecord.put("billnum", currentBillnum);
+ failedRecord.put("cgjsdid", currentCgjsdid);
+ failedRecord.put("errorMessage", ee.getMessage());
+ failedRecord.put("errorType", ee.getClass().getName());
+ failedRecords.add(failedRecord);
+ }
+ }
+ }
+
+ try {
+ rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
+ rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
+ rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
+ rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
+ rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
+ } catch (Exception ex) {
+ ex.printStackTrace();
+ result.put("updateError", "第" + pageResult.page + "页 关联字段更新异常：" + ex.getMessage());
+ }
+
+ if (pageCount == 0) {
+ stopFlag.set(true);
+ }
+ }
+ }
+
+ if (!stopFlag.get()) {
+ if (currentCap < MAX_THREAD_COUNT) {
+ currentCap++;
+ }
+ while (!stopFlag.get() && activeTasks < currentCap) {
+ if (!submitPageTask(completionService, nextPage, stopFlag, api, dwbh, pageSize)) {
+ break;
+ }
+ activeTasks++;
+ }
+ }
+ }
+ } catch (Exception e) {
+ e.printStackTrace();
+ result.put("error", "增量同步数据异常: " + e.getMessage());
+ } finally {
+ executor.shutdownNow();
+ }
+
+ rebuildPermissions(affectedIds, tablename);
+
+ String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 并发处理完成";
+ result.put("updateInfo", syncInfo);
+ result.put("diffBillnums", diffBillnums);
+ result.put("newBillnums", newBillnums);
+ result.put("updatedDetails", updatedDetails);
+ if (failCount > 0) {
+ result.put("failedRecords", failedRecords);
+ result.put("failCount", failCount);
+ }
+ if (debugLogs.size() > 0) {
+ result.put("debugLogs", debugLogs);
+ }
+ return result;
+}
*** End Patch
<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8"%>
<%@ page import="com.alibaba.fastjson.JSONArray" %>
<%@ page import="com.alibaba.fastjson.JSONObject" %>
<%@ page import="weaver.general.Util" %>
<%@ page import="dev.soft.lb.util.CommonUtil" %>
<%@ page import="dev.soft.lb.util.LCUtil" %>
<%@ page import="weaver.general.TimeUtil" %>
<%@ page import="weaver.conn.RecordSet" %>
<%@ page import="weaver.hrm.company.SubCompanyComInfo" %>
<%@ page import="weaver.formmode.setup.ModeRightInfo" %>
<%@ page import="java.text.SimpleDateFormat" %>
<%@ page import="java.util.HashMap" %>
<%@ page import="java.util.Map" %>
<%@ page import="java.math.BigDecimal" %>
<%@ page import="java.math.RoundingMode" %>
<%@ page import="java.util.Set" %>
<%@ page import="java.util.HashSet" %>
<%@ page import="java.util.Arrays" %>
<%@ page import="java.util.UUID" %>
<%@ page import="java.util.concurrent.Callable" %>
<%@ page import="java.util.concurrent.CompletionService" %>
<%@ page import="java.util.concurrent.ExecutorCompletionService" %>
<%@ page import="java.util.concurrent.ExecutorService" %>
<%@ page import="java.util.concurrent.Executors" %>
<%@ page import="java.util.concurrent.Future" %>
<%@ page import="java.util.concurrent.atomic.AtomicBoolean" %>
<%@ page import="java.util.concurrent.atomic.AtomicInteger" %>
<%
// 设置响应内容类型为JSON
response.setContentType("application/json;charset=UTF-8");
// 允许跨域访问
response.setHeader("Access-Control-Allow-Origin", "*");
response.setHeader("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
response.setHeader("Access-Control-Allow-Headers", "Content-Type, Authorization");
// 定义常量
final String companyId = "66";
final String billtype = "12";
String api = "";
final String tablename = "uf_cgjsd";
final SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
final SimpleDateFormat sdf1 = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
// 创建返回结果
JSONObject result = new JSONObject();
try {
// 获取单位编码
SubCompanyComInfo scci = new SubCompanyComInfo();
String dwbm = scci.getSubCompanyCode(companyId);
// 获取API配置
RecordSet rs = new RecordSet();
Map<String, String> billconfig = LCUtil.getLCBillTypeInfo(billtype, rs);
api = billconfig.get("api_list");
// 先做一次去重（不清空表，做增量同步）
// deduplicateDatabase(rs, tablename);
// 构建请求参数 - 分页增量拉取
JSONObject paramJson = new JSONObject();
paramJson.put("dwbh", dwbm);
paramJson.put("pagesize", "10000");
paramJson.put("page", "1");
// 执行增量同步
JSONObject syncResult = syncData(paramJson, rs, api, tablename, sdf);
result.put("status", 1);
result.put("data", syncResult);
result.put("msg", "增量同步采购结算单成功");
// 仅保留核心关心内容（差异单据编号billnum、新增单据编号billnum）在顶层返回，便于前端直接消费
if (syncResult.containsKey("diffBillnums")) {
result.put("diffBillnums", syncResult.getJSONArray("diffBillnums"));
}
if (syncResult.containsKey("newBillnums")) {
result.put("newBillnums", syncResult.getJSONArray("newBillnums"));
}
// 返回每条更新的具体字段变化（字段名、旧值、新值）
if (syncResult.containsKey("updatedDetails")) {
result.put("updatedDetails", syncResult.getJSONArray("updatedDetails"));
}
// 添加详细的同步信息
if (syncResult.containsKey("updateInfo")) {
result.put("updateInfo", syncResult.getString("updateInfo"));
}
if (syncResult.containsKey("updateError")) {
result.put("updateError", syncResult.getString("updateError"));
}
// 添加失败记录信息
if (syncResult.containsKey("failedRecords")) {
result.put("failedRecords", syncResult.getJSONArray("failedRecords"));
}
if (syncResult.containsKey("failCount")) {
result.put("failCount", syncResult.getInteger("failCount"));
}
} catch (Exception e) {
e.printStackTrace();
result.put("status", 0);
result.put("msg", "读取浪潮采购结算单异常：" + e.getMessage());
result.put("errorDetail", e.toString());
}
// 输出JSON结果
out.print(result.toJSONString());
/**
* 同步数据方法
*/
%>
<%!
private static final int DEFAULT_USER_ID = 1;
private static final int INITIAL_THREAD_COUNT = 10;
private static final int MAX_THREAD_COUNT = 20;
private static class PageResult {
final int page;
final JSONObject response;
PageResult(int page, JSONObject response) {
this.page = page;
this.response = response;
}
}
/**
* 数据库去重方法 - 删除重复数据，保留时间最新的一条
*/
public void deduplicateDatabase(RecordSet rs, String tablename) {
try {
String sql = "DELETE t1 FROM " + tablename + " t1 " +
"INNER JOIN " + tablename + " t2 " +
"WHERE t1.id < t2.id " +
"AND t1.cgjsdid = t2.cgjsdid " +
"AND t1.billnum = t2.billnum " +
"AND t1.WLDW = t2.WLDW " +
"AND t1.DEPARTMENTID = t2.DEPARTMENTID " +
"AND t1.contractid = t2.contractid " +
"AND t1.XMID = t2.XMID " +
"AND t1.Kxxz = t2.Kxxz " +
"AND t1.amount = t2.amount " +
"AND t1.Billdate = t2.Billdate";
rs.execute(sql);
} catch (Exception e) {
e.printStackTrace();
}
}
// 数值字段名单：这些字段按两位小数口径进行比较与存储
private static final Set<String> NUMERIC_FIELDS = new HashSet<>(Arrays.asList(
"amount",
"foreigncurrency",
"clearamount",
"clearforeigncurrency",
"balanceamount",
"balanceforeigncurrency",
"availableamount",
// "availableforeigncurrency", // 表中不存在此字段
// 添加其他可能的数值字段
"original_clearamount",
"original_clearforeigncurrency",
"original_balanceamount",
"original_balanceforeigncurrency",
"price",
"unitprice",
"quantity",
"taxrate",
"taxamount",
"totalmoney",
"netamount",
"amountnotax"
));
// 规范化字符串：去首尾空格；null按空串处理
private static String norm(String v) {
if (v == null) return "";
return v.trim();
}
// 字符串等价：规范化后比较；空串与null等价
private static boolean equalNormalized(String a, String b) {
return norm(a).equals(norm(b));
}
// 数值等价：两值都按两位小数（HALF_UP）规整后比较；均为空视为相等
private static boolean equalNumeric(String a, String b) {
String sa = norm(a);
String sb = norm(b);
if (sa.isEmpty() && sb.isEmpty()) return true;
if (sa.isEmpty() || sb.isEmpty()) return false;
try {
BigDecimal ba = new BigDecimal(sa).setScale(2, RoundingMode.HALF_UP);
BigDecimal bb = new BigDecimal(sb).setScale(2, RoundingMode.HALF_UP);
return ba.compareTo(bb) == 0;
} catch (Exception ignore) {
return sa.equals(sb);
}
}
// 将数值字符串格式化为两位小数；非法数字则返回规范化后的原值
private static String formatTo2Decimals(String v) {
String s = norm(v);
if (s.isEmpty()) return s;
try {
return new BigDecimal(s).setScale(2, RoundingMode.HALF_UP).toPlainString();
} catch (Exception ignore) {
return s;
}
}
// 判断数据库中的记录与新数据是否存在差异（以数据库字段为准，对比所有字段）
public boolean hasDataChanged(RecordSet rs, Map<String, String> newData) {
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return true; // 无法获取字段信息，默认认为有变更
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
System.out.println("[字段差异] " + columnName + ": 数据库值='" + dbValue + "', 接口值='" + newValue + "'");
return true; // 发现差异
}
}
return false; // 所有字段都相同
} catch (Exception e) {
e.printStackTrace();
// 出现异常时，默认认为有变更，以保证数据一致性
return true;
}
}
// 收集与数据库记录相比的字段差异，返回每个差异字段的 {field, oldValue, newValue}
public JSONArray collectDifferences(RecordSet rs, Map<String, String> newData) {
JSONArray changes = new JSONArray();
try {
// 获取数据库记录的所有字段名
String[] columnNames = rs.getColumnName();
if (columnNames == null || columnNames.length == 0) {
return changes; // 无法获取字段信息
}
// 遍历数据库的所有字段进行对比
for (String columnName : columnNames) {
// 跳过系统字段
if ("id".equalsIgnoreCase(columnName) || "modedatacreater".equalsIgnoreCase(columnName)
|| "modedatacreatertype".equalsIgnoreCase(columnName) || "modedatacreatetime".equalsIgnoreCase(columnName)
|| "modedatamodifier".equalsIgnoreCase(columnName) || "modedatamodifytime".equalsIgnoreCase(columnName)) {
continue;
}
String dbValue;
try {
dbValue = Util.null2String(rs.getString(columnName));
} catch (Exception ignore) {
continue; // 无法获取该字段值，跳过
}
// 仅对新数据中存在的字段进行比较；避免将未提供的新字段误判为空串
if (!newData.containsKey(columnName)) {
continue;
}
// 获取新数据中对应字段的值
String newValue = Util.null2String(newData.get(columnName));
// 根据字段类型进行对比
boolean equal = NUMERIC_FIELDS.contains(columnName.toLowerCase()) ?
equalNumeric(dbValue, newValue) : equalNormalized(dbValue, newValue);
if (!equal) {
JSONObject diff = new JSONObject();
diff.put("field", columnName);
diff.put("oldValue", dbValue);
diff.put("newValue", newValue);
changes.add(diff);
}
}
} catch (Exception e) {
// 忽略，返回当前已收集到的差异
}
return changes;
}
// 先对比后插入/更新：按 (cgjsdid,billnum) 定位；差异则按原id更新；不存在则插入
public int processRecordCompareAndUpsert(Map<String, String> data, RecordSet rs, String tablename,
JSONArray updateDetails, JSONArray failedRecords, Set<Integer> affectedIds) {
String billnum = Util.null2String(data.get("billnum"));
String cgjsdid = Util.null2String(data.get("cgjsdid"));
try {
RecordSet check = new RecordSet();
check.executeQuery("SELECT * FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?", cgjsdid, billnum);
if (check.next()) {
int existingId = check.getInt("id");
if (hasDataChanged(check, data)) {
JSONArray changes = collectDifferences(check, data);
if (updateDetails != null) {
JSONObject detail = new JSONObject();
detail.put("billnum", billnum);
detail.put("cgjsdid", cgjsdid);
detail.put("changes", changes);
updateDetails.add(detail);
}
if (changes != null && changes.size() > 0) {
if (applyUpdate(existingId, changes, tablename)) {
if (affectedIds != null) {
affectedIds.add(existingId);
}
return 1;
}
throw new RuntimeException("更新采购结算单失败");
}
}
return 0;
}
Integer newId = insertRecord(data, tablename);
if (newId != null && newId > 0) {
if (affectedIds != null) {
affectedIds.add(newId);
}
return 2;
}
throw new RuntimeException("新增采购结算单失败");
} catch (Exception e) {
if (failedRecords != null) {
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", billnum);
failedRecord.put("cgjsdid", cgjsdid);
failedRecord.put("errorMessage", e.getMessage());
failedRecord.put("operation", "upsert");
failedRecords.add(failedRecord);
}
return -1;
}
}
private boolean applyUpdate(int recordId, JSONArray changes, String tablename) {
if (recordId <= 0 || changes == null || changes.isEmpty()) {
return false;
}
StringBuilder setClause = new StringBuilder();
for (int i = 0; i < changes.size(); i++) {
JSONObject change = changes.getJSONObject(i);
String field = Util.null2String(change.getString("field"));
String value = Util.null2String(change.getString("newValue"));
if (field.isEmpty()) {
continue;
}
if (setClause.length() > 0) {
setClause.append(",");
}
setClause.append(field).append("=").append(toSqlValue(value));
}
if (setClause.length() == 0) {
return false;
}
String sql = "UPDATE " + tablename + " SET " + setClause + " WHERE id=" + recordId;
RecordSet updateRs = new RecordSet();
return updateRs.execute(sql);
}
private Integer insertRecord(Map<String, String> data, String tablename) {
if (data == null || data.isEmpty()) {
return null;
}
Map<String, String> insertMap = new HashMap<>(data);
String now = TimeUtil.getCurrentTimeString();
String createDate = now.substring(0, 10);
String createTime = now.substring(11);
insertMap.put("formmodeid", String.valueOf(getModeId(tablename)));
insertMap.put("modedatacreater", String.valueOf(DEFAULT_USER_ID));
insertMap.put("modedatacreatertype", "1");
insertMap.put("modedatacreatedate", createDate);
insertMap.put("modedatacreatetime", createTime);
insertMap.put("modeuuid", UUID.randomUUID().toString());
StringBuilder columns = new StringBuilder();
StringBuilder values = new StringBuilder();
for (Map.Entry<String, String> entry : insertMap.entrySet()) {
if (columns.length() > 0) {
columns.append(",");
values.append(",");
}
columns.append(entry.getKey());
values.append(toSqlValue(entry.getValue()));
}
RecordSet insertRs = new RecordSet();
String sql = "INSERT INTO " + tablename + "(" + columns + ") VALUES(" + values + ")";
if (!insertRs.execute(sql)) {
return null;
}
RecordSet idRs = new RecordSet();
idRs.executeQuery("SELECT id FROM " + tablename + " WHERE cgjsdid = ? AND billnum = ?",
Util.null2String(data.get("cgjsdid")), Util.null2String(data.get("billnum")));
if (idRs.next()) {
return idRs.getInt("id");
}
return null;
}
private String toSqlValue(String value) {
String normalized = Util.null2String(value);
if (normalized.isEmpty()) {
return "null";
}
return "'" + normalized.replace("'", "''") + "'";
}
private void rebuildPermissions(Set<Integer> affectedIds, String tablename) {
if (affectedIds == null || affectedIds.isEmpty()) {
return;
}
try {
int modeId = getModeId(tablename);
if (modeId <= 0) {
return;
}
ModeRightInfo rightInfo = new ModeRightInfo();
rightInfo.setNewRight(false);
rightInfo.init();
rightInfo.setModeId(modeId);
for (Integer billId : affectedIds) {
if (billId == null || billId <= 0) {
continue;
}
rightInfo.rebuildModeDataShareByEdit(DEFAULT_USER_ID, modeId, billId);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
private int getModeId(String tablename) {
try {
return Util.getIntValue(CommonUtil.getModeIdByTableName(tablename));
} catch (Exception e) {
return 0;
}
}
private boolean submitPageTask(CompletionService<PageResult> completionService,
AtomicInteger nextPage,
AtomicBoolean stopFlag,
String api,
String dwbh,
String pageSize) {
if (stopFlag.get()) {
return false;
}
completionService.submit(new Callable<PageResult>() {
@Override
public PageResult call() throws Exception {
if (stopFlag.get()) {
return null;
}
int pageIndex = nextPage.getAndIncrement();
JSONObject req = new JSONObject();
req.put("dwbh", dwbh);
req.put("pagesize", pageSize);
req.put("page", String.valueOf(pageIndex));
JSONObject resp = LCUtil.doPost(api, req);
if (resp != null && resp.containsKey("code") && resp.getIntValue("code") == 3003) {
stopFlag.set(true);
}
return new PageResult(pageIndex, resp);
}
});
return true;
}
public JSONObject syncData(JSONObject param, RecordSet rs, String api, String tablename, SimpleDateFormat sdf) {
JSONObject result = new JSONObject();
int totalCount = 0;
int newCount = 0;
int updateCount = 0;
int skipCount = 0;
int failCount = 0;
JSONArray diffBillnums = new JSONArray();
JSONArray newBillnums = new JSONArray();
JSONArray updatedDetails = new JSONArray();
JSONArray failedRecords = new JSONArray();
JSONArray debugLogs = new JSONArray();
Set<Integer> affectedIds = new HashSet<>();
String dwbh = Util.null2String(param.getString("dwbh"));
String pageSize = Util.null2String(param.getString("pagesize"));
if ("".equals(pageSize)) pageSize = "10000";
int startPage = Integer.parseInt(Util.null2String(param.getString("page")).equals("") ? "1" : param.getString("page"));
AtomicInteger nextPage = new AtomicInteger(startPage);
AtomicBoolean stopFlag = new AtomicBoolean(false);
ExecutorService executor = Executors.newFixedThreadPool(MAX_THREAD_COUNT);
CompletionService<PageResult> completionService = new ExecutorCompletionService<>(executor);
int currentCap = INITIAL_THREAD_COUNT;
int activeTasks = 0;
try {
for (int i = 0; i < currentCap; i++) {
if (submitPageTask(completionService, nextPage, stopFlag, api, dwbh, pageSize)) {
activeTasks++;
}
}
while (activeTasks > 0) {
Future<PageResult> future = completionService.take();
PageResult pageResult = null;
try {
pageResult = future.get();
} catch (Exception taskEx) {
failCount++;
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", "");
failedRecord.put("cgjsdid", "");
failedRecord.put("operation", "thread");
failedRecord.put("errorMessage", "线程执行异常: " + taskEx.getMessage());
failedRecord.put("errorType", taskEx.getClass().getName());
failedRecords.add(failedRecord);
}
activeTasks--;
if (pageResult != null && pageResult.response != null && !stopFlag.get()) {
JSONObject rjson = pageResult.response;
if (rjson.containsKey("code") && rjson.getIntValue("code") == 3003) {
stopFlag.set(true);
} else if (!(rjson.containsKey("code") && rjson.getIntValue("code") == 30000)) {
stopFlag.set(true);
result.put("updateError", "第" + pageResult.page + "页接口返回异常code: " + rjson.getString("code"));
} else {
JSONArray datas = rjson.getJSONArray("data");
int pageCount = (datas == null) ? 0 : datas.size();
totalCount += pageCount;
System.out.println("多线程处理 - 第 " + pageResult.page + " 页获取到 " + pageCount + " 条数据");
if (datas != null) {
for (int x = 0; x < datas.size(); x++) {
JSONObject item = datas.getJSONObject(x);
Map<String, String> data = new HashMap<>();
String currentBillnum = "";
String currentCgjsdid = "";
try {
currentCgjsdid = Util.null2String(item.get("ID"));
currentBillnum = Util.null2String(item.get("BILLNUM"));
data.put("cgjsdid", currentCgjsdid);
data.put("billnum", currentBillnum);
data.put("WLDW", Util.null2String(item.get("WLDW")));
data.put("partnerid_code", Util.null2String(item.get("PARTNERID_CODE")));
data.put("PARTNERID_NAME", Util.null2String(item.get("PARTNERID_NAME")));
data.put("DEPARTMENTID", Util.null2String(item.get("DEPARTMENTID")));
data.put("departmentid_name", Util.null2String(item.get("DEPARTMENTID_NAME")));
data.put("BMBH", Util.null2String(item.get("BMBH")));
data.put("billtypeid", Util.null2String(item.get("BILLTYPEID")));
data.put("summary", Util.null2String(item.get("SUMMARY")));
data.put("billcateprop", Util.null2String(item.get("BILLCATEPROP")));
data.put("DJLXMC", Util.null2String(item.get("DJLXMC")));
data.put("contractid", Util.null2String(item.get("CONTRACTID")));
data.put("contractid_code", Util.null2String(item.get("CONTRACTID_CODE")));
data.put("contractid_name", Util.null2String(item.get("CONTRACTID_NAME")));
data.put("currencyid", Util.null2String(item.get("CURRENCYID")));
data.put("organizationid", Util.null2String(item.get("ORGANIZATIONID")));
data.put("dwbh", Util.null2String(item.get("DWBH")));
data.put("organizationid_name", Util.null2String(item.get("ORGANIZATIONID_NAME")));
data.put("customtext01", Util.null2String(item.get("CUSTOMTEXT01")));
data.put("employeeid", Util.null2String(item.get("EMPLOYEEID")));
data.put("employeeid_name", Util.null2String(item.get("EMPLOYEEID_NAME")));
data.put("employeeid_code", Util.null2String(item.get("EMPLOYEEID_CODE")));
data.put("Kxxz", Util.null2String(item.get("KXXZ")));
data.put("KxxzFullMC", Util.null2String(item.get("KXXZFULLMC")));
data.put("Kxxzno", Util.null2String(item.get("KXXZNO")));
data.put("KXXZMC", Util.null2String(item.get("KXXZMC")));
data.put("XMID", Util.null2String(item.get("XMID")));
data.put("projectinfoid_code", Util.null2String(item.get("PROJECTINFOID_CODE")));
data.put("projectinfoid_name", Util.null2String(item.get("PROJECTINFOID_NAME")));
data.put("billstatename", Util.null2String(item.get("BILLSTATENAME")));
data.put("srcbizid", Util.null2String(item.get("SRCBIZID")));
if (item.get("BILLDATE") != null) {
data.put("Billdate", sdf.format(item.getDate("BILLDATE")));
}
data.put("srcdoctypeid", Util.null2String(item.get("SRCDOCTYPEID")));
data.put("srcdoctypecode", Util.null2String(item.get("SRCDOCTYPECODE")));
data.put("amount", formatTo2Decimals(item.getString("AMOUNT")));
data.put("foreigncurrency", formatTo2Decimals(item.getString("FOREIGNCURRENCY")));
data.put("clearamount", formatTo2Decimals(item.getString("ORIGINAL_CLEARAMOUNT")));
data.put("clearforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_CLEARFOREIGNCURRENCY")));
data.put("balanceamount", formatTo2Decimals(item.getString("ORIGINAL_BALANCEAMOUNT")));
data.put("balanceforeigncurrency", formatTo2Decimals(item.getString("ORIGINAL_BALANCEFOREIGNCURRENCY")));
data.put("availableamount", formatTo2Decimals(item.getString("AVAILABLEAMOUNT")));
data.put("amountnotax", formatTo2Decimals(item.getString("AMOUNTNOTAX")));
int action = processRecordCompareAndUpsert(data, rs, tablename, updatedDetails, failedRecords, affectedIds);
if (action == 1) {
updateCount++;
diffBillnums.add(data.get("billnum"));
} else if (action == 2) {
newCount++;
newBillnums.add(data.get("billnum"));
} else if (action == -1) {
failCount++;
} else {
skipCount++;
}
} catch (Exception ee) {
failCount++;
JSONObject failedRecord = new JSONObject();
failedRecord.put("billnum", currentBillnum);
failedRecord.put("cgjsdid", currentCgjsdid);
failedRecord.put("errorMessage", ee.getMessage());
failedRecord.put("errorType", ee.getClass().getName());
failedRecords.add(failedRecord);
}
}
}
try {
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbwldw B ON A.WLDW = B.nm SET A.wldwmcll = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xjhttz B ON A.contractid = B.gscbizid SET A.htmc = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_esbkxxz B ON A.kxxz = B.nm SET A.kxxz1 = B.id");
rs.execute("UPDATE uf_cgjsd A INNER JOIN uf_xmba B ON A.xmid = B.gscbizid SET A.xmmc = B.id");
rs.execute("UPDATE uf_cgjsd u INNER JOIN uf_hrmdepartment h1 ON h1.bm = u.bmbh INNER JOIN hrmdepartment h2 ON h2.departmentcode = h1.bm SET u.bmmcll = h2.id");
} catch (Exception ex) {
ex.printStackTrace();
result.put("updateError", "第" + pageResult.page + "页 关联字段更新异常：" + ex.getMessage());
}
if (pageCount == 0) {
stopFlag.set(true);
}
}
}
if (!stopFlag.get()) {
if (currentCap < MAX_THREAD_COUNT) {
currentCap++;
}
while (!stopFlag.get() && activeTasks < currentCap) {
if (!submitPageTask(completionService, nextPage, stopFlag, api, dwbh, pageSize)) {
break;
}
activeTasks++;
}
}
}
} catch (Exception e) {
e.printStackTrace();
result.put("error", "增量同步数据异常: " + e.getMessage());
} finally {
executor.shutdownNow();
}
rebuildPermissions(affectedIds, tablename);
String syncInfo = "增量同步统计 - 总数:" + totalCount + ", 新增:" + newCount + ", 更新:" + updateCount + ", 跳过:" + skipCount + ", 失败:" + failCount + "; 并发处理完成";
result.put("updateInfo", syncInfo);
result.put("diffBillnums", diffBillnums);
result.put("newBillnums", newBillnums);
result.put("updatedDetails", updatedDetails);
if (failCount > 0) {
result.put("failedRecords", failedRecords);
result.put("failCount", failCount);
}
if (debugLogs.size() > 0) {
result.put("debugLogs", debugLogs);
}
return result;
}
/**
* 调试辅助：打印 ModeUtil.saveModeData 可能使用的表信息
*/
public String debugTableInfo(String tablename, RecordSet rs) {
StringBuilder info = new StringBuilder();
try {
// 获取表的列信息
rs.executeQuery("SHOW COLUMNS FROM " + tablename);
info.append("[表结构信息] ").append(tablename).append(":\n");
while (rs.next()) {
String field = rs.getString("Field");
String type = rs.getString("Type");
String nullAllowed = rs.getString("Null");
String key = rs.getString("Key");
String defaultVal = rs.getString("Default");
info.append(" ").append(field).append(" ").append(type);
if ("NO".equals(nullAllowed)) info.append(" NOT NULL");
if ("PRI".equals(key)) info.append(" PRIMARY KEY");
if (defaultVal != null) info.append(" DEFAULT ").append(defaultVal);
info.append("\n");
}
} catch (Exception e) {
info.append("获取表结构失败: ").append(e.getMessage());
}
return info.toString();
}
%>
关键调整
syncLCCGJSD.jsp现在在页面加载时自动构建 10 个并发任务，利用ExecutorCompletionService管控最多 20 个线程。每个任务仅抓取一个分页并返回；一旦某个线程收到code=3003或空页，立刻设置stopFlag，阻止继续发起新请求，并等现有线程处理完再shutdown。- 为线程调度新增
AtomicInteger nextPage和AtomicBoolean stopFlag。初始并发为 10，若仍未碰到 3003，会在任务完成后逐步把上限提升到 20；超过 20 时不会再提交，等这批线程处理完毕自动销毁。 - 所有接口响应仍在主线程串行落库：单页数据解析、
processRecordCompareAndUpsert、五条关联更新 SQL 以及差异统计逻辑保持不变，最后统一收集affectedIds并执行权限重构。 - 新增辅助
PageResult类和submitPageTask方法，隔离接口调用与同步逻辑，同时把dwbh/pagesize/page组装放在任务内，确保线程间互不干扰。
使用提示
- 默认
pagesize仍为 10000，可按前面测算改成 3000~5000 平衡内存与接口耗时；修改后多线程调度会自动生效。 - 目前仍是“抢占式”请求，接口若无法承受 10~20 并发，请把
INITIAL_THREAD_COUNT和MAX_THREAD_COUNT下调。 - JSP 环境未易于做全量集成测试，建议在测试环境先跑一次 10 万条级别数据，观察接口响应是否因并发升高而报错。
User
请求并行然后数据处理多线程是否是更好的解决方案
Cursor
并行拉取 + 多线程写库看似“更快”，但对你这个场景风险大于收益：
- 接口瓶颈在网络/上游：每次 3‑4 s 是 API 固有延迟，多线程只是同时排队等待，无法把单页响应压缩到<1 s；更高并发还可能触发流控或 429。我们已经让 JSP 最多 20 并发请求，基本把上游资源吃满了，再加本地多线程处理意义不大。
- 本地“写库”已经是批量 SQL：目前单页结果在 JSP 中串行落库、批量更新，再统一重构权限。瓶颈主要在 IO，而不是 CPU。如果把
processRecordCompareAndUpsert改为多线程同时执行，会导致RecordSet/DB 连接被抢、同一条记录的写入冲突、甚至死锁。 - 线程调度成本：JSP 运行在 Web 容器里，创建大量 worker 需要额外线程池和事务管理。如果既要接口并行又要处理并行，就得做好全局锁、连接池、事务隔离，否则偶发问题更难排查。
更稳健的方向
- 继续优化分页条件：引入“增量时间戳/最大 ID”，让每次只处理近 10 分钟数据；真实 workload 会瞬间降几个量级。
- 把写库放到 DB 侧批处理：把接口数据写临时表，SQL 一次性
INSERT ... SELECT/UPDATE ... JOIN，既减少 Java 内存，又让 DB 自己处理并发。- 必要时并行请求：已经实现 10→20 并发线程调度，足以压榨接口吞吐；如果仍慢，优先调大
pagesize而不是把数据处理再拆多线程。
总结：并行请求 + 单线程落库 是目前的较优折中，再叠加数据处理多线程会显著增加复杂度和风险，收益不明显。建议把精力放在更小 pagesize、增量条件、数据库批更新等可控方案上。
User
是否有最优解
Cursor
没有“放之四海皆准”的最优解，只有结合现状的最合适策略。就你这个采购结算同步场景，综合接口性能、JSP 运行环境和数据量，我认为较优的路线是：
-
接口层：保持中等分页 + 并发拉取
pagesize控制在 3k5k：单页内存 <10 MB，100k 数据 ≈ 2034 页。- 维持 10→20 线程的阶梯并发：既能利用上游带宽，又不过度压垮接口。
- 一旦任意线程拿到
code=3003或空页立即停发，避免多余请求。
-
数据层：单线程批处理 + DB 原子操作
- JSP 端收集一页数据后串行执行批量 SQL（我们已将
processRecordCompareAndUpsert调整为直接 UPDATE/INSERT），避免多线程抢同一记录。 - 每页处理完再做关联更新，末尾统一重构权限。若后面瓶颈仍在写库，可进一步改成“接口结果写临时表 ➜ DB 侧
INSERT/UPDATE ... SELECT”，让数据库自己并行化。
- JSP 端收集一页数据后串行执行批量 SQL（我们已将
-
增量策略：真正减少数据量
- 结合业务字段（
Billdate、状态、max id）增加接口条件，只同步近 10 分钟或未入库记录。这样不需要 10 万条“全量对比”，性能提升比任何并发/多线程都大。 - 若上游暂不支持，先落地我们已有的并发方案，在测试环境测出真实耗时，再推动接口方提供增量参数。
- 结合业务字段（
所以当前版本的“并发请求 + 单线程批处理 + 统一权限重构”已经是现阶段较优折中。再去同时把数据处理做多线程，很难保证数据库一致性且收益有限。真正的“最优”，是让每次任务只处理少量增量 + DB 端批处理，用结构化设计解决问题，而不是盲目堆线程。
User
这里做对比的业务是用于更新历史数据内容
Cursor
理解了：当前同步逻辑并非只拉“最近变更”，而是要不断对比/更新历史数据，保证本地 uf_cgjsd 与浪潮端保持一致。所以：
- 我们在 JSP/定时任务里做的是“全字段对比 + 差异更新”，即便记录是几个月前的历史单，也会重新对齐金额或状态。这也是为什么
processRecordCompareAndUpsert要拉取整页数据后，再逐字段比较、只更新变化字段。 - 这种模式适合处理上游补录、回冲或历史纠错的场景，但成本就是每轮都要扫描大量记录。之前提到的增量优化，也可以做“混合模式”：
- 常规调度：只处理近期新增/修改（速度快）。
- 历史修复批任务：按天/月分批重刷，或像现在这样“全量对比”，但在低频窗口（比如每日夜间）执行。
- 如果历史更新是稀有事件，也可以考虑给下游加“变更标记”或“待对账列表”，只针对有异常的历史记录触发重新同步，不必每次都全量对比。
总之，以“更新历史数据”为目标，目前的字段对比+差异更新逻辑可以满足，但要注意调度窗口、接口压力和数据库资源，必要时把历史刷新拆成后台批处理，以免影响每 10 分钟的常规同步。
User
这里历史数据更新不是稀有情况
Cursor
如果历史数据被频繁回写/补录，那“常规增量 + 偶发全量”策略就不够用了，这时最好建立一个长期可运行的历史对账机制，但仍要控制资源消耗。可考虑：
- 按时间片滚动对账：把历史分段（例如按月份、按项目），每次任务仅处理一个片区，循环轮询。这样一次仍是“全字段对比”，但每轮只扫一部分历史，既能不断修正旧数据，又不会每 10 分钟都扫 10 万条。
- 基于上游的“最近更新时间”或版本号：如果浪潮接口能返回
lastModified，即便是历史单，只要它修改过仍会被拉到本次任务，真正避免遗漏；目前你每次全量拉取的做法就是在模拟这种机制，但量太大，可以推动上游提供更新时间字段。 - 维护本地“疑似需重新同步”队列：当下游系统检测到历史字段异常或业务人员手动触发时，把单据 ID 推进重同步队列，任务优先处理这些 ID，之后再做普通增量。这样虽然复杂一些，但能保证高频历史更新不会被遗漏，同时避免每次都扫全表。
换句话说，既然历史变更很常见，就应当正式设计“历史数据修复”流程，而不是永久依赖“全量对比”。当前的并发版本能暂时支撑，但为了长期稳定，建议尽快落地上述分段对账或队列机制，减轻接口和数据库压力。
User
多线程不行，中间会掉数据
Cursor
是的，直接把多页请求并行拉取再在 JSP 中落库，确实有几类“丢数据”风险：
- 接口层本身无序：线程 A、B 同时请求相邻页，如果上游那边刚好新增/删除数据，返回的数据会重排或跳页，可能导致某些记录被跳过或重复。
- 落库阶段没有事务边界：我们在 JSP 里依次更新数据库，如果多线程一起写，可能出现记录 A 被线程1更新后，线程2又根据旧数据覆盖回去，造成数据丢失或回滚。
- 关联更新 & 权限重构是全局操作：并发线程同时执行关联 SQL、重建权限，会互相覆盖或锁住表，部分线程失败时就需要重试，否则就会遗漏该批数据。
- 失败重试难以追踪：单线程时出错能精确定位页码和记录；多线程同时跑时，一旦某个线程挂掉或接口 3003 先返回，其他线程可能继续处理后续页，导致当页数据直接丢失。
所以我们在最近的修改里把“多线程并发”限制在只对接口请求，数据落库仍保持单线程批处理。这样即使请求顺序打乱，处理时仍按每个线程返回的页逐条对比，不会互相覆盖或漏掉记录。如果你继续观察到数据缺失，建议：
- 先回退到单线程（把
INITIAL_THREAD_COUNT/MAX_THREAD_COUNT调成 1），确认是否稳定； - 若单线程正常，多线程有问题，说明接口分页在并行情况下无法保证结果一致，这就需要维持单线程拉取，或让接口端提供“游标/版本号”。
- 另外可以在 JSP 里为每页增加日志和校验（比如统计 billnum 集合），在重跑时对比，确认是否真的漏了某页。
总之，多线程只适合提升吞吐，不适合在缺乏强一致保障的历史对账场景里直接上。当前版本中如需完全避免风险，可把线程数调回 1，并结合增量/分段对账等方式减少整体耗时。
User
一个线程也不行，pass多线程